-- Logs begin at Mon 2022-08-08 01:28:00 UTC, end at Mon 2022-08-08 02:54:38 UTC. -- Aug 08 02:11:58.035102 dv-cinder-733062-33-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 08 02:11:58.043592 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 08 02:11:58.045591 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 8 02:11:58 2022] *** Aug 08 02:11:58.045591 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 08 02:11:58.045591 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 08 02:11:58.045591 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: nodename: dv-cinder-733062-33-iscsi Aug 08 02:11:58.045591 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: machine: x86_64 Aug 08 02:11:58.045591 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: clock source: unix Aug 08 02:11:58.045591 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: pcre jit disabled Aug 08 02:11:58.045591 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: detected number of CPU cores: 4 Aug 08 02:11:58.045591 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: current working directory: / Aug 08 02:11:58.045591 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: detected binary path: /usr/bin/uwsgi-core Aug 08 02:11:58.046087 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: your processes number limit is 47680 Aug 08 02:11:58.046087 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: your memory page size is 4096 bytes Aug 08 02:11:58.046087 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: detected max file descriptor number: 2048 Aug 08 02:11:58.046087 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: lock engine: pthread robust mutexes Aug 08 02:11:58.046087 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: thunder lock: enabled Aug 08 02:11:58.046087 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 08 02:11:58.046087 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 08 02:11:58.085725 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: Python main interpreter initialized at 0x558491b0f9c0 Aug 08 02:11:58.085725 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: python threads support enabled Aug 08 02:11:58.085725 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: your server socket listen backlog is limited to 100 connections Aug 08 02:11:58.085725 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: your mercy for graceful operations on workers is 90 seconds Aug 08 02:11:58.086065 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: mapped 403077 bytes (393 KB) for 2 cores Aug 08 02:11:58.086137 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: *** Operational MODE: preforking *** Aug 08 02:11:58.086433 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: *** uWSGI is running in multiple interpreter mode *** Aug 08 02:11:58.086433 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: spawned uWSGI master process (pid: 80146) Aug 08 02:11:58.086624 dv-cinder-733062-33-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 08 02:11:58.087671 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: spawned uWSGI worker 1 (pid: 80159, cores: 1) Aug 08 02:11:58.087671 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: spawned uWSGI worker 2 (pid: 80160, cores: 1) Aug 08 02:11:58.087671 dv-cinder-733062-33-iscsi devstack@keystone.service[80146]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 08 02:12:03.558444 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:12:03.558444 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: from cryptography.utils import int_from_bytes Aug 08 02:12:03.559451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:12:03.559451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: from cryptography.utils import int_from_bytes Aug 08 02:12:03.559570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:12:03.559570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: from cryptography.utils import int_from_bytes Aug 08 02:12:03.559570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:12:03.559570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: from cryptography.utils import int_from_bytes Aug 08 02:12:03.609179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 02:12:03.609318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 02:12:03.609409 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 02:12:03.609484 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 02:12:03.609542 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] command line args: [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 02:12:03.609649 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] command line args: [] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 02:12:03.609782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 02:12:03.609934 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 02:12:03.610028 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ================================================================================ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 02:12:03.610100 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ================================================================================ {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 02:12:03.610332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] admin_token = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.610497 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] admin_token = **** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.610576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config_dir = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.610748 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] config_dir = [] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.610832 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.610939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.611016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config_source = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.611125 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] config_source = [] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.611208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] debug = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.611307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] debug = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.611391 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.611625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.611720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.611778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.611902 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.612006 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.612153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.612263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.612404 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.612576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.612687 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] list_limit = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.612787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] log_config_append = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.612879 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.612978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.613084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_config_append = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.613179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] log_dir = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.613270 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.613367 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] log_file = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.613473 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_dir = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.613584 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.613708 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.613783 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.613901 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.613981 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.614204 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.614315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.614424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.614534 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.614618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.614832 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.614911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.615035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.615112 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.615238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.615319 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.615440 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.615518 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.615724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.615805 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.615963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.616079 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.616238 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.616329 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.616489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.616617 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.616767 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.616894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.617165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.617243 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.617419 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] publish_errors = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.617498 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.617651 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.617840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.617932 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.618047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.618321 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.618444 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.618556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.618644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] publish_errors = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.618782 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] standard_threads = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.618925 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.619026 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.619214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.619292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.619408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.619490 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.619603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] use_journal = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.619789 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.619871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] use_json = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.619941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] use_stderr = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.620008 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.620076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] use_syslog = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.620279 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] standard_threads = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.620372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.620552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.620634 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.620792 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.620869 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.621050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.621136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.621256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_journal = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.621592 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.621675 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_json = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.621744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.621818 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_stderr = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.621897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.621968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_syslog = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.622042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.622117 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:12:03.622215 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.622453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.622544 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.622730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.622807 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.622981 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.623100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.623218 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.623414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.623493 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.623695 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.623770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.623952 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.624064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.624203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.624432 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.624509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.624690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.624867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.624958 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.625180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.625257 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.625417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.625559 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.625672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.625808 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.625960 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.626033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.626175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.626299 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.626451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.626530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.626685 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.626770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.626917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.626994 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.627148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.627230 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.627383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.627461 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.627632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.627711 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.627881 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.627958 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.628113 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.628190 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.628344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.628426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.628591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.628668 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.628840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.628918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.629079 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.629156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.629332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.629454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.629544 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.629618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.629775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.629872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.630012 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.630111 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.630242 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.630360 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.630497 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.630586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.630739 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.630825 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.630968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.631052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.631202 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.631284 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.631450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.631603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.631676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.631747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.632433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.632520 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.632669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.632756 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.632905 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.632983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.633137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.633212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.633368 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.633452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.633598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.633676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.633828 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.633936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.634057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.634169 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.634297 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.634399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.634549 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.634651 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.634778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.634895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.635036 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.635127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.635233 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.635309 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.635459 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.635547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.635688 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.635766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.635928 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.636012 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.636175 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.636291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.636399 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.636482 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.636633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.636708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.636869 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.636959 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.637106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.637187 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.637338 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.637418 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.637568 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.637664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.637814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.637895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.638049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.638124 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.638307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.638413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.638542 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.638641 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.638783 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.638880 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.639026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.639114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.639254 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.639336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.639491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.639570 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.639726 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.639813 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.639962 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.640048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.640201 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.640282 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.640438 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.640517 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.640678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.640757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.640913 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.640993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.641148 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.641246 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.641380 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.641461 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.641626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.641706 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.641864 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.641941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.642098 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.642180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.642361 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.642443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.642597 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.642687 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.642832 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.642918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.643060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.643138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.643292 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.643369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.643518 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.643592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.643743 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.643822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.643974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.644059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.644219 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.644352 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.644438 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.644510 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.644679 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.644768 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.644915 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.645009 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.645147 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.645230 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.645385 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.645472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.645611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.645691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.645839 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.645920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] auth.external = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.646067 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.646146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.646322 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.646418 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.646553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.646635 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.646795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.646874 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] auth.password = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.647031 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.647118 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] auth.token = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.647280 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.647360 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.648058 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.648150 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.648323 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.648413 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.648560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.648653 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.648812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.648894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.649042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.external = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.649121 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.649273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.649362 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.649509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] credential.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.649589 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.649759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.649841 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.650005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.password = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.650093 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.650234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.token = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.650330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.650494 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.650581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.650731 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.650826 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.650965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.651046 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.651195 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.651275 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.651441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.651530 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.651690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.651777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.651921 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.652006 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.652155 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.652245 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.652403 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.652484 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.652650 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.652732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.652897 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.652985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.653134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.653216 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.653377 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.653460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] federation.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.653611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.653700 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.653874 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.653963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.654127 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.654207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.654384 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.654479 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.654636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.654717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.654878 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.654960 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.655137 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.655233 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.655377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.655458 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.655625 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.655706 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.655862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.655950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.656101 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.656180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.656336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.656428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.656579 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.656668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.656826 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.656913 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.657054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.657133 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.657288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.657376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.657521 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.657601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.657760 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.657845 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.657995 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.658071 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.658236 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.658339 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.658500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.658582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.658741 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.658822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.658973 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.659052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.659203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.659284 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.659452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.659605 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.659692 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.659765 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.659902 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.659983 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.660147 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.660230 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.660378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.660459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.660615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.660695 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.660861 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.660943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.661111 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.661192 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.661342 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.661423 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.661572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.661654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.661803 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.661887 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.662041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.662122 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.662312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.662400 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.662552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.662634 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.662803 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.662883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.663054 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.663136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.663311 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.663391 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.663563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.663644 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.663798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.663879 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.664024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.664104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.664260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.664340 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.664489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.664572 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.664722 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.664803 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.664957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.665036 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.665193 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.665274 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.665430 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.665509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.665665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.665744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.665902 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.665982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.666141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.666221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.666395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.666481 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.666642 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.666725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.666877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.666956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.667104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.667183 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.667341 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.667424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.667570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.667651 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.667806 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.667887 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.668040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.668121 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.668275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.668355 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.668507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.668587 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.668739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.668820 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.668987 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.669068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.669219 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.669298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.669453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.669534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.669681 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.669762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.669927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.670016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.670162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.670241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.670419 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.670500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.670652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.670745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.670886 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.670967 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.671117 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.671201 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.671348 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.671435 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.671600 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.671688 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.671828 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.671908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.672072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.672161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.672317 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.672397 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.672561 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.672641 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.672791 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.672878 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.673017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.673096 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.673249 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.673330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.673480 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.673569 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.673715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.673795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.673939 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.674018 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.674183 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.674294 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.674453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.674534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.674695 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.674774 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.674935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.675023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.675173 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.675253 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.675398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.675478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.675643 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.675730 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.675876 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.675956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.676110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.676189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.676340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.676428 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.676574 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.676655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.676811 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.676890 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.677050 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.677140 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.677284 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.677363 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] resource.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.677533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.677613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.677761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.677848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.678004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.678086 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.678240 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.678344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.678497 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.678591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.678759 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.678840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.679007 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.679089 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.679248 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.679338 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.679478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.679559 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.679713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.679792 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] role.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.679950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.680040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] role.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.680180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.680260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.680432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.680512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.680663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.680819 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.680896 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.680968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.681147 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.681229 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.681380 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.681470 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.681626 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.681708 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.681859 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.681939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.682098 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.682193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.682354 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.682436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.682600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.682679 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.682830 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.682914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.683076 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.683158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.683321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.683403 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.683569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.683722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.683802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.683959 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.684039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.684498 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.684594 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.684750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.684831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.685001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.685081 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.685234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.685328 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.685475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.685555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.685717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.685836 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.685963 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.686095 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.686207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.686363 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.686601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.686680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.686836 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.686928 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.687071 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.687152 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.687317 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.687396 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.687573 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.687661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.687825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.687906 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.688064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.688144 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.688303 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.688385 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.688538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.688625 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.688776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] token.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.688857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.689016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.689097 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.689266 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.689354 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.689502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.689582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.689754 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.689837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.690002 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.690081 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.690241 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.690376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.690498 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.690580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.690735 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.690816 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.690965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.691046 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.691199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.691281 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.691437 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.691516 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.691674 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.691754 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.691906 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.691987 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.692156 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.692242 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.692389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.692470 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.692622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.692703 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.692866 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.692947 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.693106 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.693186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.connection = **** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.693352 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.693443 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.693590 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.693672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.693828 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.693908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.694062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.694143 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.694886 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.695025 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.695101 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.695172 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.695310 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.695392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.695545 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.695627 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.695793 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.695874 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.696031 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.696111 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.696275 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.696369 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.696510 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.696639 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.696760 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.696842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.696999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.697088 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.697234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.697315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.697464 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.697543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.697696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.697785 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.697929 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.698010 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.698083 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 02:12:03.698156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.698417 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.698663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.698898 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.699150 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.699384 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.699627 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.699856 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.700094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.700327 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:12:03.700497 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 02:12:03.727505 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.727740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.727970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.728605 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.728835 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.729066 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.737621 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.738629 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.766034 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.766267 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.766647 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.766877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.818506 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.818644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.868586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.868812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.869545 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.869780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.933394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.933622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.933843 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.933928 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.934060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.934461 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.934551 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.934627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.934758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.934843 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.934993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.935211 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.957900 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.958086 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.958215 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.958368 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.958464 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.958542 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.958689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.958774 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.958912 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.959005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.959130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.959214 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.959345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.959562 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.970073 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.970465 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.982711 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.982850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.982937 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.983084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.983185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.983311 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.983396 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.983535 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.983618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.983755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.983840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.983962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.984044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:03.984177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:12:04.001183 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.001394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 02:12:04.001736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.001819 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.001935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 02:12:04.002019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 02:12:04.002210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.002376 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.002497 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.002581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 02:12:04.002759 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.002858 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.003015 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.003098 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.003336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.003596 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.014877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.015155 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.015420 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.015504 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.015699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.015786 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.015972 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.016262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.022589 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.023189 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.025632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.025917 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.026207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.026318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.026529 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.026613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.026818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.026914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.027103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.027188 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.027384 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.027467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.027649 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.027733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.027918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.027999 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.028193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.028274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.028457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.028540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.028734 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.029006 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.051412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.051771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.052904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.053197 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.053281 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.053517 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.058307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.058592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.058677 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.058876 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.058961 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.059453 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.065641 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.065928 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.066013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.066303 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.070487 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.070778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.070974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.071059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.071272 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.071356 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.071561 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.071645 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.071847 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.071930 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.072123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.072207 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.072390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.072477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.072687 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.072967 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.087916 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.088185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.088793 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.089079 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.092083 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.093083 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.095225 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.095522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.096248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.096550 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.099878 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.100963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.103087 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.104137 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.106195 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.106517 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.106816 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.107111 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.107304 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.107389 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.107607 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.107700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.107897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.108195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.108476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.108768 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.124019 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.124308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.124574 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.124834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.125203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.125500 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.125771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.126039 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.131730 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.133450 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.133854 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.134134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.135020 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.135315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.136933 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.137219 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.137518 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.137807 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.138082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.138166 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.138415 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.138725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.139022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.139307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.149421 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.149705 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.149988 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.150291 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.150577 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.150716 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.150856 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.151004 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.151133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.151291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.151579 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.151869 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.152160 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.152445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.167532 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.168899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.170493 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.171863 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.173397 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.174820 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.174990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.176412 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.176544 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.176737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.177028 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.177907 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.178205 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.178525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.183928 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.185693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.185832 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.187322 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.188703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.188995 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.189290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.189570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.190269 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.190567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.190858 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.191136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.198059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.198383 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.198668 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.199806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.200100 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.200383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.205670 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.205958 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.206263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.206553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.206830 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.207105 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.207235 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:12:04.207392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.207524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.207657 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.207797 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.208064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.208204 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.208338 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.208481 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.208627 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.208761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.208892 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.209042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.209225 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.209359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.209491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.209680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.209810 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.210041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.210343 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.210619 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.210886 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:12:04.242734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558491b0f9c0 pid: 80160 (default app) Aug 08 02:12:04.244415 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558491b0f9c0 pid: 80159 (default app) Aug 08 02:12:04.245931 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a90b1-f48c-431d-b7e0-807400889f7f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:04.246212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a90b1-f48c-431d-b7e0-807400889f7f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:04.246505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a90b1-f48c-431d-b7e0-807400889f7f None None] PATH_INFO: `/v3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:04.250768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1/1] 10.222.0.39 () {50 vars in 791 bytes} [Mon Aug 8 02:12:04 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:11.844288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079dcb0-753b-4620-88d9-02b3d816f3bc None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:11.844880 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079dcb0-753b-4620-88d9-02b3d816f3bc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:11.844880 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079dcb0-753b-4620-88d9-02b3d816f3bc None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:11.849304 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1/2] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:12:11.854406 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8be3dd-368f-468a-8691-44bf0dd5ad97 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:11.854836 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8be3dd-368f-468a-8691-44bf0dd5ad97 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:11.854945 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8be3dd-368f-468a-8691-44bf0dd5ad97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:11.925550 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: INFO dbcounter [None req-ff8be3dd-368f-468a-8691-44bf0dd5ad97 None None] Registered counter for database keystone Aug 08 02:12:11.945227 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG oslo_db.sqlalchemy.engines [None req-ff8be3dd-368f-468a-8691-44bf0dd5ad97 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80160) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:12:11.951364 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writer thread running {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:12:12.128207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG passlib.handlers.bcrypt [None req-ff8be3dd-368f-468a-8691-44bf0dd5ad97 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80160) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 02:12:12.128544 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG passlib.handlers.bcrypt [None req-ff8be3dd-368f-468a-8691-44bf0dd5ad97 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80160) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 02:12:12.183061 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ff8be3dd-368f-468a-8691-44bf0dd5ad97 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:12.184944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ff8be3dd-368f-468a-8691-44bf0dd5ad97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:12.312942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 2/3] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 08 02:12:12.457674 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079dcb0-753b-4620-88d9-02b3d816f3bc None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:12.527190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: INFO dbcounter [None req-b079dcb0-753b-4620-88d9-02b3d816f3bc None None] Registered counter for database keystone Aug 08 02:12:12.548414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG oslo_db.sqlalchemy.engines [None req-b079dcb0-753b-4620-88d9-02b3d816f3bc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80159) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:12:12.555104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writer thread running {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:12:12.670495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079dcb0-753b-4620-88d9-02b3d816f3bc None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:12.674707 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0901af-dfdd-41ae-beca-08db12cc0686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:12.675319 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0901af-dfdd-41ae-beca-08db12cc0686 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:12.675595 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0901af-dfdd-41ae-beca-08db12cc0686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:12.675864 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0901af-dfdd-41ae-beca-08db12cc0686 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:12.684645 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0901af-dfdd-41ae-beca-08db12cc0686 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:12.735535 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4b0901af-dfdd-41ae-beca-08db12cc0686 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 02:12:12.735820 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0901af-dfdd-41ae-beca-08db12cc0686 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:12.745653 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 2/4] 10.222.0.39 () {58 vars in 1324 bytes} [Mon Aug 8 02:12:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:12:12.751564 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8be3dd-368f-468a-8691-44bf0dd5ad97 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:12.765696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8be3dd-368f-468a-8691-44bf0dd5ad97 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:12.769986 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55852b-fb19-4143-899f-7eafea911546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:12.771106 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55852b-fb19-4143-899f-7eafea911546 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:12.771388 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55852b-fb19-4143-899f-7eafea911546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:12.771659 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55852b-fb19-4143-899f-7eafea911546 admin admin] PATH_INFO: `/v3/projects/520cd26c01a04fe5b710bc3899ff38a8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:12.773114 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55852b-fb19-4143-899f-7eafea911546 admin admin] RBAC: Authorizing `identity:get_project(project_id=520cd26c01a04fe5b710bc3899ff38a8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:12.823881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1d55852b-fb19-4143-899f-7eafea911546 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 02:12:12.824218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55852b-fb19-4143-899f-7eafea911546 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:12.828844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 3/5] 10.222.0.39 () {56 vars in 1207 bytes} [Mon Aug 8 02:12:12 2022] GET /identity/v3/projects/520cd26c01a04fe5b710bc3899ff38a8 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:14.769728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bd577-7560-4851-83a3-0298b79059f9 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:14.770268 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bd577-7560-4851-83a3-0298b79059f9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:14.770368 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bd577-7560-4851-83a3-0298b79059f9 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:14.774939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 3/6] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:14.780281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f03596-a234-4a9c-ae16-57b0fbb2768e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:14.780673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f03596-a234-4a9c-ae16-57b0fbb2768e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:14.780837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f03596-a234-4a9c-ae16-57b0fbb2768e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:14.802205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-25f03596-a234-4a9c-ae16-57b0fbb2768e None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:14.803748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-25f03596-a234-4a9c-ae16-57b0fbb2768e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:14.811837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 4/7] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 08 02:12:14.833777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bd577-7560-4851-83a3-0298b79059f9 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:14.846365 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bd577-7560-4851-83a3-0298b79059f9 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:14.850535 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a101a-c72f-497c-bbb0-f4d8d445547c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:14.851119 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a101a-c72f-497c-bbb0-f4d8d445547c admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:14.851395 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a101a-c72f-497c-bbb0-f4d8d445547c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:14.851665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a101a-c72f-497c-bbb0-f4d8d445547c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:14.859864 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a101a-c72f-497c-bbb0-f4d8d445547c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:14.861621 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a101a-c72f-497c-bbb0-f4d8d445547c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:14.870130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 4/8] 10.222.0.39 () {58 vars in 1324 bytes} [Mon Aug 8 02:12:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:12:14.876242 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f03596-a234-4a9c-ae16-57b0fbb2768e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:14.885697 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f03596-a234-4a9c-ae16-57b0fbb2768e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:14.890018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a706a-3f18-40c4-aaac-37dfc02848d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:14.890668 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a706a-3f18-40c4-aaac-37dfc02848d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:14.890951 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a706a-3f18-40c4-aaac-37dfc02848d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:14.891227 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a706a-3f18-40c4-aaac-37dfc02848d5 admin admin] PATH_INFO: `/v3/users/2d716e3acbb54b2fa8e10b30ba8eb5fb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:14.892551 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a706a-3f18-40c4-aaac-37dfc02848d5 admin admin] RBAC: Authorizing `identity:get_user(user_id=2d716e3acbb54b2fa8e10b30ba8eb5fb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:14.894401 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a706a-3f18-40c4-aaac-37dfc02848d5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:14.895892 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 5/9] 10.222.0.39 () {56 vars in 1198 bytes} [Mon Aug 8 02:12:14 2022] GET /identity/v3/users/2d716e3acbb54b2fa8e10b30ba8eb5fb => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:16.945062 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48917bd-b804-4a89-a9a7-007852ece011 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:16.945596 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48917bd-b804-4a89-a9a7-007852ece011 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:16.945680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48917bd-b804-4a89-a9a7-007852ece011 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:16.946984 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 5/10] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:12:16.950016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138b2e8-d4d2-4d3a-b184-b8f67bc3627c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:16.950858 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138b2e8-d4d2-4d3a-b184-b8f67bc3627c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:16.951295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138b2e8-d4d2-4d3a-b184-b8f67bc3627c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:16.952189 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aea928-6056-4583-aece-03221907ce36 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:16.952475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aea928-6056-4583-aece-03221907ce36 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:16.952623 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 6/11] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:12:16.952729 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aea928-6056-4583-aece-03221907ce36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:16.957698 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46fa6f-b80c-4c4b-9715-b1f93821818d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:16.957979 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46fa6f-b80c-4c4b-9715-b1f93821818d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:16.958280 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46fa6f-b80c-4c4b-9715-b1f93821818d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:16.976978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7c46fa6f-b80c-4c4b-9715-b1f93821818d None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:16.978148 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7c46fa6f-b80c-4c4b-9715-b1f93821818d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:16.997771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG passlib.handlers.bcrypt [None req-42aea928-6056-4583-aece-03221907ce36 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80159) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 02:12:16.998323 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG passlib.handlers.bcrypt [None req-42aea928-6056-4583-aece-03221907ce36 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80159) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 02:12:17.044167 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-42aea928-6056-4583-aece-03221907ce36 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:17.045787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-42aea928-6056-4583-aece-03221907ce36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:17.059613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 7/12] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:12:17.081299 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46fa6f-b80c-4c4b-9715-b1f93821818d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:17.106177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:17.150407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 6/13] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:17.172020 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aea928-6056-4583-aece-03221907ce36 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:17.189724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46fa6f-b80c-4c4b-9715-b1f93821818d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:17.225772 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b8d31-0515-4b81-9c6e-cc758d8eeb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:17.226343 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b8d31-0515-4b81-9c6e-cc758d8eeb87 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:17.226627 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b8d31-0515-4b81-9c6e-cc758d8eeb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:17.226897 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b8d31-0515-4b81-9c6e-cc758d8eeb87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:17.272125 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aea928-6056-4583-aece-03221907ce36 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:17.275061 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:17.293374 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b8d31-0515-4b81-9c6e-cc758d8eeb87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:17.294893 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b8d31-0515-4b81-9c6e-cc758d8eeb87 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:17.309712 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b551bb-0753-40bc-9779-740345fc9132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:17.310853 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b551bb-0753-40bc-9779-740345fc9132 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:17.311140 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b551bb-0753-40bc-9779-740345fc9132 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:17.311414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b551bb-0753-40bc-9779-740345fc9132 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:17.317788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:17.323619 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b551bb-0753-40bc-9779-740345fc9132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:17.331968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-b1b551bb-0753-40bc-9779-740345fc9132 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:12:17.331968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:17.331968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:17.331968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:17.331968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:17.331968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:17.331968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:17.331968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:17.331968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:17.331968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:17.331968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:17.331968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:17.331968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:17.331968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:17.331968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:17.331968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:17.333022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:17.333022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:17.333022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:17.333022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:17.333022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:17.333022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:17.333022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:17.333022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:17.333022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:17.333022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:17.333022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:17.333022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:17.333022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:17.333022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:17.333022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:17.333022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:17.333806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:17.333806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:17.333806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:17.333806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:17.333806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:17.333806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:17.333806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:17.333806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:17.333806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:17.333806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:17.333806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:12:17.333806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:12:17.333806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 7/14] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:12:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:12:17.341404 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b551bb-0753-40bc-9779-740345fc9132 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:17.396302 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 8/15] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:12:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 02:12:17.402517 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b8d31-0515-4b81-9c6e-cc758d8eeb87 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:17.442965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b551bb-0753-40bc-9779-740345fc9132 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:17.481896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f0273-e7ab-46cf-9a37-52409641fd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:17.482470 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f0273-e7ab-46cf-9a37-52409641fd8f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:17.482752 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f0273-e7ab-46cf-9a37-52409641fd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:17.483022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f0273-e7ab-46cf-9a37-52409641fd8f None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:17.483689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f0273-e7ab-46cf-9a37-52409641fd8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:17.483992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f0273-e7ab-46cf-9a37-52409641fd8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:17.485422 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f0273-e7ab-46cf-9a37-52409641fd8f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:17.499023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 8/16] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:17.500732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b8d31-0515-4b81-9c6e-cc758d8eeb87 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:17.504721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f0273-e7ab-46cf-9a37-52409641fd8f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:17.523951 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:17.531104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:17.535981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a447669-9fe9-4d20-8aeb-ee79ca2e6821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:17.536604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a447669-9fe9-4d20-8aeb-ee79ca2e6821 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:17.536881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a447669-9fe9-4d20-8aeb-ee79ca2e6821 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:17.537145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a447669-9fe9-4d20-8aeb-ee79ca2e6821 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:17.547695 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a447669-9fe9-4d20-8aeb-ee79ca2e6821 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:17.549172 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a447669-9fe9-4d20-8aeb-ee79ca2e6821 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:17.568071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5a447669-9fe9-4d20-8aeb-ee79ca2e6821 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:12:17.568071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:17.568071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:17.568071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:17.568071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:17.568071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:17.568071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:17.568071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:17.568071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:17.568071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:17.568071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:17.568071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:17.568071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:12:17.568071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:12:17.568071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:12:17.568071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:12:17.568968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:17.568968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:17.568968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:17.568968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:17.568968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:17.568968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:17.568968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:17.568968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:17.568968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:17.568968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:17.568968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:17.568968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:17.568968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:17.568968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:17.568968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:17.568968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:17.568968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:12:17.569757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:12:17.569757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:12:17.569757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:17.569757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 9/17] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:12:17.574842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a447669-9fe9-4d20-8aeb-ee79ca2e6821 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:17.605914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f0273-e7ab-46cf-9a37-52409641fd8f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:17.663455 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72035f5-d9f1-4ee6-bf65-adcb8c06a322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:17.664197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72035f5-d9f1-4ee6-bf65-adcb8c06a322 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:17.664600 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72035f5-d9f1-4ee6-bf65-adcb8c06a322 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:17.664979 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72035f5-d9f1-4ee6-bf65-adcb8c06a322 None admin] PATH_INFO: `/v3/users/2d716e3acbb54b2fa8e10b30ba8eb5fb` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:17.667309 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72035f5-d9f1-4ee6-bf65-adcb8c06a322 None admin] RBAC: Authorizing `identity:get_user(user_id=2d716e3acbb54b2fa8e10b30ba8eb5fb)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:17.669528 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72035f5-d9f1-4ee6-bf65-adcb8c06a322 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:17.671737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 9/18] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:17 2022] GET /identity/v3/users/2d716e3acbb54b2fa8e10b30ba8eb5fb => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:17.678096 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72035f5-d9f1-4ee6-bf65-adcb8c06a322 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:17.775271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:17.780610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a447669-9fe9-4d20-8aeb-ee79ca2e6821 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:17.782226 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72035f5-d9f1-4ee6-bf65-adcb8c06a322 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:17.813303 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa32e5-4e8f-4d01-9e1d-484a1bb419a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:17.813896 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa32e5-4e8f-4d01-9e1d-484a1bb419a0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:17.814174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa32e5-4e8f-4d01-9e1d-484a1bb419a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:17.814469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa32e5-4e8f-4d01-9e1d-484a1bb419a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:17.815104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa32e5-4e8f-4d01-9e1d-484a1bb419a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:17.815403 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa32e5-4e8f-4d01-9e1d-484a1bb419a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:17.815606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5e2ba-5a91-4d1a-868f-1f4ac6662fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:17.816224 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5e2ba-5a91-4d1a-868f-1f4ac6662fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:17.816500 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5e2ba-5a91-4d1a-868f-1f4ac6662fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:17.816768 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5e2ba-5a91-4d1a-868f-1f4ac6662fa9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:17.816851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa32e5-4e8f-4d01-9e1d-484a1bb419a0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:17.817868 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5e2ba-5a91-4d1a-868f-1f4ac6662fa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:17.819815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5e2ba-5a91-4d1a-868f-1f4ac6662fa9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:17.821363 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 10/19] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:12:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:17.826994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5e2ba-5a91-4d1a-868f-1f4ac6662fa9 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:17.834045 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 10/20] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:17.929754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5e2ba-5a91-4d1a-868f-1f4ac6662fa9 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:17.966790 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8c20f-5bb9-47ea-9535-f76a543c13d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:17.967496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8c20f-5bb9-47ea-9535-f76a543c13d4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:17.967756 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8c20f-5bb9-47ea-9535-f76a543c13d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:17.968032 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8c20f-5bb9-47ea-9535-f76a543c13d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:17.968767 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8c20f-5bb9-47ea-9535-f76a543c13d4 None admin] RBAC: Adding query filter params (role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.domain.id=default, user.id=2d716e3acbb54b2fa8e10b30ba8eb5fb) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:17.969073 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8c20f-5bb9-47ea-9535-f76a543c13d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:17.970634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8c20f-5bb9-47ea-9535-f76a543c13d4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:17.980184 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 11/21] 10.222.0.39 () {56 vars in 1322 bytes} [Mon Aug 8 02:12:17 2022] GET /identity/v3/role_assignments?user.id=2d716e3acbb54b2fa8e10b30ba8eb5fb&scope.domain.id=default&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:20.174708 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9f27a-642f-4357-b161-23be80ebd22a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:20.175243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9f27a-642f-4357-b161-23be80ebd22a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:20.175243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9f27a-642f-4357-b161-23be80ebd22a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:20.176783 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 11/22] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:20.182129 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc599fe9-5e1c-409e-a8b7-2ea71acf6513 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:20.182599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc599fe9-5e1c-409e-a8b7-2ea71acf6513 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:20.182735 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc599fe9-5e1c-409e-a8b7-2ea71acf6513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:20.206675 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-dc599fe9-5e1c-409e-a8b7-2ea71acf6513 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:20.207874 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dc599fe9-5e1c-409e-a8b7-2ea71acf6513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:20.232307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:20.285519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 12/23] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:20.320360 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9f27a-642f-4357-b161-23be80ebd22a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:20.362513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:20.455070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9f27a-642f-4357-b161-23be80ebd22a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:20.504833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e352d7-1000-4a34-b6ff-a5402218350e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:20.505467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e352d7-1000-4a34-b6ff-a5402218350e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:20.505747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e352d7-1000-4a34-b6ff-a5402218350e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:20.506026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e352d7-1000-4a34-b6ff-a5402218350e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:20.516343 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e352d7-1000-4a34-b6ff-a5402218350e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:20.517830 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e352d7-1000-4a34-b6ff-a5402218350e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:20.533220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c3e352d7-1000-4a34-b6ff-a5402218350e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:12:20.533220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:20.533220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:20.533220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:20.533220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:20.533220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:20.533220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:20.533220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:20.533220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:20.533220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:20.533220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:20.533220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:20.533220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:12:20.533220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:12:20.533220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:12:20.533220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:12:20.534791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:20.534791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:20.534791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:20.534791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:20.534791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:20.534791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:20.534791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:20.534791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:20.534791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:20.534791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:20.534791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:20.534791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:20.534791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:20.534791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:20.534791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:20.534791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:20.534791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:12:20.535586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:12:20.535586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:12:20.535586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:20.535586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 12/24] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:20.541078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc599fe9-5e1c-409e-a8b7-2ea71acf6513 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:20.633104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fe96f-911d-4622-b075-afbc700ebaf1 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:20.633417 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fe96f-911d-4622-b075-afbc700ebaf1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:20.633645 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fe96f-911d-4622-b075-afbc700ebaf1 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:20.635007 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 13/25] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:12:20.645856 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c997e-4182-4a07-b387-27021201784f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:20.646138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c997e-4182-4a07-b387-27021201784f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:20.646423 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c997e-4182-4a07-b387-27021201784f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:20.666291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc599fe9-5e1c-409e-a8b7-2ea71acf6513 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:20.683284 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-420c997e-4182-4a07-b387-27021201784f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:20.684974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-420c997e-4182-4a07-b387-27021201784f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:20.703019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37726d90-55b0-42aa-8cf6-49cdf75f475a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:20.703697 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37726d90-55b0-42aa-8cf6-49cdf75f475a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:20.704243 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37726d90-55b0-42aa-8cf6-49cdf75f475a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:20.704654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37726d90-55b0-42aa-8cf6-49cdf75f475a None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:20.705928 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37726d90-55b0-42aa-8cf6-49cdf75f475a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:20.706355 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37726d90-55b0-42aa-8cf6-49cdf75f475a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:20.708256 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37726d90-55b0-42aa-8cf6-49cdf75f475a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:20.737570 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 13/26] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:20.738863 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432013e-4e11-4615-9c36-21fab20b0190 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:20.739150 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432013e-4e11-4615-9c36-21fab20b0190 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:20.739427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432013e-4e11-4615-9c36-21fab20b0190 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:20.740611 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 14/27] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:20.751125 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432013e-4e11-4615-9c36-21fab20b0190 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:20.789788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 14/28] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:20.791102 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028aaf1-f18c-47e2-83d4-ead9ed84c5b6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:20.791390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028aaf1-f18c-47e2-83d4-ead9ed84c5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:20.791664 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028aaf1-f18c-47e2-83d4-ead9ed84c5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:20.811350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4028aaf1-f18c-47e2-83d4-ead9ed84c5b6 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:20.812564 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4028aaf1-f18c-47e2-83d4-ead9ed84c5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:20.830277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:20.918272 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 15/29] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:20.918272 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028aaf1-f18c-47e2-83d4-ead9ed84c5b6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:20.938377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:21.023020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028aaf1-f18c-47e2-83d4-ead9ed84c5b6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:21.058704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148093f-43b2-4904-ad15-68a03cd15d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:21.059651 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148093f-43b2-4904-ad15-68a03cd15d76 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:21.059934 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148093f-43b2-4904-ad15-68a03cd15d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:21.060246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148093f-43b2-4904-ad15-68a03cd15d76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:21.072776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148093f-43b2-4904-ad15-68a03cd15d76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:21.079395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5148093f-43b2-4904-ad15-68a03cd15d76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:12:21.079395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:21.079395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:21.079395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:21.079395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:21.079395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:21.079395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:21.079395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:21.079395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:21.079395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:21.079395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:21.079395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:21.079395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:21.079395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:21.079395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:21.079395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:21.080231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:21.080231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:21.080231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:21.080231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:21.080231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:21.080231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:21.080231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:21.080231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:21.080231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:21.080231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:21.080231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:21.080231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:21.080231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:21.080231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:21.080231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:21.080231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:21.080954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:21.080954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:21.080954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:21.080954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:21.080954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:21.080954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:21.080954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:21.080954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:21.080954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:21.080954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:21.080954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:12:21.080954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:21.080954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 16/30] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:12:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:21.081572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148093f-43b2-4904-ad15-68a03cd15d76 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:21.095433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:21.108529 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432013e-4e11-4615-9c36-21fab20b0190 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:21.151543 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe363c-acdb-47d3-8043-f142d0ba1cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:21.152171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe363c-acdb-47d3-8043-f142d0ba1cea None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:21.152766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe363c-acdb-47d3-8043-f142d0ba1cea None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:21.153051 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe363c-acdb-47d3-8043-f142d0ba1cea None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:21.153705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe363c-acdb-47d3-8043-f142d0ba1cea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:21.155983 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe363c-acdb-47d3-8043-f142d0ba1cea None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:21.206917 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148093f-43b2-4904-ad15-68a03cd15d76 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:21.244210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e4d77-978f-4acc-87b5-5b4457763e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:21.245021 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e4d77-978f-4acc-87b5-5b4457763e7c None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:21.245431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e4d77-978f-4acc-87b5-5b4457763e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:21.245834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e4d77-978f-4acc-87b5-5b4457763e7c None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:21.246633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e4d77-978f-4acc-87b5-5b4457763e7c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:21.248174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e4d77-978f-4acc-87b5-5b4457763e7c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:21.328726 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 15/31] 10.222.0.39 () {60 vars in 1141 bytes} [Mon Aug 8 02:12:20 2022] POST /identity/v3/projects => generated 301 bytes in 578 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:12:21.330116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c953d-562a-4d19-8879-bdd6d0ae3e16 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:21.330447 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c953d-562a-4d19-8879-bdd6d0ae3e16 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:21.330725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c953d-562a-4d19-8879-bdd6d0ae3e16 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:21.332053 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 16/32] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:21.332860 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c953d-562a-4d19-8879-bdd6d0ae3e16 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:21.381278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 17/33] 10.222.0.39 () {60 vars in 1131 bytes} [Mon Aug 8 02:12:21 2022] POST /identity/v3/roles => generated 225 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:12:21.383661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600bbfb-a1b3-4206-bfc7-9251e5c2e17f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:21.384022 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600bbfb-a1b3-4206-bfc7-9251e5c2e17f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:21.384375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600bbfb-a1b3-4206-bfc7-9251e5c2e17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:21.441608 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8600bbfb-a1b3-4206-bfc7-9251e5c2e17f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:21.443310 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8600bbfb-a1b3-4206-bfc7-9251e5c2e17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:21.573875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:21.594783 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:21.602412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 18/34] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:21.607650 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d72a8-8032-42da-81bd-6b99afe06793 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:21.607943 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d72a8-8032-42da-81bd-6b99afe06793 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:21.608223 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d72a8-8032-42da-81bd-6b99afe06793 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:21.610899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 19/35] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:12:21.617321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564c0ad-9150-4c00-a92f-fd078448d742 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:21.618965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564c0ad-9150-4c00-a92f-fd078448d742 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:21.619247 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564c0ad-9150-4c00-a92f-fd078448d742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:21.629747 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c953d-562a-4d19-8879-bdd6d0ae3e16 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:21.642562 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1564c0ad-9150-4c00-a92f-fd078448d742 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:21.644496 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1564c0ad-9150-4c00-a92f-fd078448d742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:21.685145 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae68de5-e30b-4ea1-b380-29c9e637334f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:21.685777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae68de5-e30b-4ea1-b380-29c9e637334f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:21.686056 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae68de5-e30b-4ea1-b380-29c9e637334f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:21.686359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae68de5-e30b-4ea1-b380-29c9e637334f None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:21.687043 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae68de5-e30b-4ea1-b380-29c9e637334f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:21.687352 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae68de5-e30b-4ea1-b380-29c9e637334f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:21.688976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae68de5-e30b-4ea1-b380-29c9e637334f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:21.701009 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 17/36] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:21.702948 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae68de5-e30b-4ea1-b380-29c9e637334f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:21.745674 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 20/37] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:12:21.746674 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564c0ad-9150-4c00-a92f-fd078448d742 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:21.826208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae68de5-e30b-4ea1-b380-29c9e637334f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:21.844857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564c0ad-9150-4c00-a92f-fd078448d742 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:21.853435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:21.859281 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46929ca-714d-4b99-a54c-3a7e414654eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:21.859885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46929ca-714d-4b99-a54c-3a7e414654eb None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:21.860166 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46929ca-714d-4b99-a54c-3a7e414654eb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:21.860443 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46929ca-714d-4b99-a54c-3a7e414654eb None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:21.861105 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46929ca-714d-4b99-a54c-3a7e414654eb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:21.862515 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46929ca-714d-4b99-a54c-3a7e414654eb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:21.872170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:21.878659 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 18/38] 10.222.0.39 () {60 vars in 1131 bytes} [Mon Aug 8 02:12:21 2022] POST /identity/v3/roles => generated 219 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:12:21.879941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f0e38-4fc1-43c5-8220-3b7d55b62e18 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:21.880215 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f0e38-4fc1-43c5-8220-3b7d55b62e18 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:21.880490 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f0e38-4fc1-43c5-8220-3b7d55b62e18 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:21.881889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 19/39] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:21.882011 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e14a0-ad93-45d4-aa91-832d17db0674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:21.882458 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f0e38-4fc1-43c5-8220-3b7d55b62e18 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:21.882625 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e14a0-ad93-45d4-aa91-832d17db0674 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:21.882908 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e14a0-ad93-45d4-aa91-832d17db0674 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:21.883243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e14a0-ad93-45d4-aa91-832d17db0674 None admin] PATH_INFO: `/v3/users/2d716e3acbb54b2fa8e10b30ba8eb5fb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:21.884458 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e14a0-ad93-45d4-aa91-832d17db0674 None admin] RBAC: Authorizing `identity:get_user(user_id=2d716e3acbb54b2fa8e10b30ba8eb5fb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:21.887178 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e14a0-ad93-45d4-aa91-832d17db0674 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:21.891371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 21/40] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:21 2022] GET /identity/v3/users/2d716e3acbb54b2fa8e10b30ba8eb5fb => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:21.893320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147968d6-42e1-429c-86fa-f266b49af128 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:21.893613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147968d6-42e1-429c-86fa-f266b49af128 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:21.893892 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147968d6-42e1-429c-86fa-f266b49af128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:21.920430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-147968d6-42e1-429c-86fa-f266b49af128 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:21.921687 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-147968d6-42e1-429c-86fa-f266b49af128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:22.003672 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 22/41] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:22.004765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147968d6-42e1-429c-86fa-f266b49af128 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:22.006084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f0e38-4fc1-43c5-8220-3b7d55b62e18 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:22.058786 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e9baa-f31b-4f71-a4c1-d66d0960ddf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:22.059657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e9baa-f31b-4f71-a4c1-d66d0960ddf1 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:22.059946 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e9baa-f31b-4f71-a4c1-d66d0960ddf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:22.060222 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e9baa-f31b-4f71-a4c1-d66d0960ddf1 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:22.060895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e9baa-f31b-4f71-a4c1-d66d0960ddf1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:22.062375 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e9baa-f31b-4f71-a4c1-d66d0960ddf1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:22.078355 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 20/42] 10.222.0.39 () {60 vars in 1131 bytes} [Mon Aug 8 02:12:21 2022] POST /identity/v3/roles => generated 223 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:12:22.079076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e9baa-f31b-4f71-a4c1-d66d0960ddf1 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:22.140644 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:22.141049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147968d6-42e1-429c-86fa-f266b49af128 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:22.169717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:22.182518 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008e4c0-dfbc-42a4-b8fa-141951695545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:22.183170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008e4c0-dfbc-42a4-b8fa-141951695545 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:22.183448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008e4c0-dfbc-42a4-b8fa-141951695545 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:22.183725 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008e4c0-dfbc-42a4-b8fa-141951695545 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:22.185138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008e4c0-dfbc-42a4-b8fa-141951695545 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:22.186679 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008e4c0-dfbc-42a4-b8fa-141951695545 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:22.188439 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 23/43] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:12:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:22.194480 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008e4c0-dfbc-42a4-b8fa-141951695545 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:22.202326 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e9baa-f31b-4f71-a4c1-d66d0960ddf1 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:22.247610 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd220e3-9817-4ab1-898f-9918cee964b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:22.248418 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd220e3-9817-4ab1-898f-9918cee964b2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:22.248704 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd220e3-9817-4ab1-898f-9918cee964b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:22.249343 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd220e3-9817-4ab1-898f-9918cee964b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:22.250706 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd220e3-9817-4ab1-898f-9918cee964b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:22.252406 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd220e3-9817-4ab1-898f-9918cee964b2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:22.253963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 21/44] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:12:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:22.259530 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd220e3-9817-4ab1-898f-9918cee964b2 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:22.309095 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008e4c0-dfbc-42a4-b8fa-141951695545 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:22.342992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030aaf5-1776-4436-bf4a-34f8e24e9b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:22.343637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030aaf5-1776-4436-bf4a-34f8e24e9b11 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:22.343914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030aaf5-1776-4436-bf4a-34f8e24e9b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:22.344187 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030aaf5-1776-4436-bf4a-34f8e24e9b11 None admin] PATH_INFO: `/v3/domains/default/users/2d716e3acbb54b2fa8e10b30ba8eb5fb/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:22.346116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1030aaf5-1776-4436-bf4a-34f8e24e9b11 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2d716e3acbb54b2fa8e10b30ba8eb5fb, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:22.347599 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1030aaf5-1776-4436-bf4a-34f8e24e9b11 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:22.361567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 24/45] 10.222.0.39 () {58 vars in 1361 bytes} [Mon Aug 8 02:12:22 2022] PUT /identity/v3/domains/default/users/2d716e3acbb54b2fa8e10b30ba8eb5fb/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:12:22.364721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd220e3-9817-4ab1-898f-9918cee964b2 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:22.383760 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:22.402535 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e55091-3b90-4579-9b1c-6c46aa0271f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:22.403070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e55091-3b90-4579-9b1c-6c46aa0271f4 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:22.403348 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e55091-3b90-4579-9b1c-6c46aa0271f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:22.403621 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e55091-3b90-4579-9b1c-6c46aa0271f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:22.404293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e55091-3b90-4579-9b1c-6c46aa0271f4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:22.405787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e55091-3b90-4579-9b1c-6c46aa0271f4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:22.437889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 22/46] 10.222.0.39 () {60 vars in 1141 bytes} [Mon Aug 8 02:12:22 2022] POST /identity/v3/projects => generated 312 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:12:24.315192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f4288-7286-4144-966c-17ddc69a4a8d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:24.315734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f4288-7286-4144-966c-17ddc69a4a8d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:24.315734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f4288-7286-4144-966c-17ddc69a4a8d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:24.317128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 25/47] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:24.322299 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fee76f-d6e0-4dac-9e1e-9b4569bc1d54 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:24.322714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fee76f-d6e0-4dac-9e1e-9b4569bc1d54 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:24.322858 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fee76f-d6e0-4dac-9e1e-9b4569bc1d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:24.341317 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b6fee76f-d6e0-4dac-9e1e-9b4569bc1d54 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:24.342834 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b6fee76f-d6e0-4dac-9e1e-9b4569bc1d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:24.416159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 23/48] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:24.438491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f4288-7286-4144-966c-17ddc69a4a8d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:24.478987 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43778813-137c-4c6c-8c2e-1967b5487ff0 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:24.479278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43778813-137c-4c6c-8c2e-1967b5487ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:24.479561 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43778813-137c-4c6c-8c2e-1967b5487ff0 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:24.486667 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 24/49] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:12:24.488321 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa0146-7405-4a84-8456-af869ee49588 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:24.488494 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa0146-7405-4a84-8456-af869ee49588 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:24.488765 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa0146-7405-4a84-8456-af869ee49588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:24.493499 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:24.507189 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-53aa0146-7405-4a84-8456-af869ee49588 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:24.508384 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-53aa0146-7405-4a84-8456-af869ee49588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:24.542700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f4288-7286-4144-966c-17ddc69a4a8d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:24.575938 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94430c8c-0d63-41d1-8aa8-e56fcb0d5679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:24.576097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 25/50] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:12:24.576488 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94430c8c-0d63-41d1-8aa8-e56fcb0d5679 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:24.576770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94430c8c-0d63-41d1-8aa8-e56fcb0d5679 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:24.577093 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94430c8c-0d63-41d1-8aa8-e56fcb0d5679 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:24.588692 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94430c8c-0d63-41d1-8aa8-e56fcb0d5679 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:24.593379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-94430c8c-0d63-41d1-8aa8-e56fcb0d5679 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:12:24.593379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:24.593379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:24.593379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:24.593379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:24.593379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:24.593379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:24.593379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:24.593379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:24.593379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:24.593379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:24.593379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:24.593379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:24.593379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:24.593379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:24.593379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:24.594179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:24.594179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:24.594179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:24.594179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:24.594179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:24.594179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:24.594179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:24.594179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:24.594179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:24.594179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:24.594179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:24.594179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:24.594179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:24.594179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:24.594179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:24.594179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:24.595103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:24.595103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:24.595103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:24.595103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:24.595103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:24.595103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:24.595103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:24.595103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:24.595103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:24.595103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:24.595103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:12:24.595103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:24.595103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 26/51] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:12:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:24.598931 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa0146-7405-4a84-8456-af869ee49588 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:24.600170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94430c8c-0d63-41d1-8aa8-e56fcb0d5679 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:24.629431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:24.701184 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa0146-7405-4a84-8456-af869ee49588 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:24.711433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94430c8c-0d63-41d1-8aa8-e56fcb0d5679 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:24.724152 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:24.735469 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b7605-b1e0-4956-b216-c896814d43c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:24.736104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b7605-b1e0-4956-b216-c896814d43c0 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:24.736385 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b7605-b1e0-4956-b216-c896814d43c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:24.736660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b7605-b1e0-4956-b216-c896814d43c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:24.737795 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429b7605-b1e0-4956-b216-c896814d43c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:24.739272 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429b7605-b1e0-4956-b216-c896814d43c0 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:24.741076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 26/52] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:12:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:24.745159 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f97015-840d-4a5a-b70c-740f66b08b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:24.745768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f97015-840d-4a5a-b70c-740f66b08b76 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:24.746051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f97015-840d-4a5a-b70c-740f66b08b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:24.746369 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f97015-840d-4a5a-b70c-740f66b08b76 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:24.746508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b7605-b1e0-4956-b216-c896814d43c0 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:24.747030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f97015-840d-4a5a-b70c-740f66b08b76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:24.747334 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f97015-840d-4a5a-b70c-740f66b08b76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:24.748717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f97015-840d-4a5a-b70c-740f66b08b76 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:24.761202 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 27/53] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:24.766493 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f97015-840d-4a5a-b70c-740f66b08b76 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:24.839461 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:24.843048 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b7605-b1e0-4956-b216-c896814d43c0 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:24.864560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f97015-840d-4a5a-b70c-740f66b08b76 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:24.875807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6ec0e-7c70-42ea-bbd6-e5f75d3a0d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:24.876412 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6ec0e-7c70-42ea-bbd6-e5f75d3a0d9c None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:24.876692 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6ec0e-7c70-42ea-bbd6-e5f75d3a0d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:24.876967 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6ec0e-7c70-42ea-bbd6-e5f75d3a0d9c None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:24.877618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6ec0e-7c70-42ea-bbd6-e5f75d3a0d9c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:24.879088 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6ec0e-7c70-42ea-bbd6-e5f75d3a0d9c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:24.902328 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f435c-f896-40a7-a4c9-4cbd07cd7d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:24.903038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f435c-f896-40a7-a4c9-4cbd07cd7d55 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:24.903323 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f435c-f896-40a7-a4c9-4cbd07cd7d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:24.903591 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f435c-f896-40a7-a4c9-4cbd07cd7d55 None admin] PATH_INFO: `/v3/users/2d716e3acbb54b2fa8e10b30ba8eb5fb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:24.904738 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f435c-f896-40a7-a4c9-4cbd07cd7d55 None admin] RBAC: Authorizing `identity:get_user(user_id=2d716e3acbb54b2fa8e10b30ba8eb5fb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:24.906187 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f435c-f896-40a7-a4c9-4cbd07cd7d55 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:24.906843 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 27/54] 10.222.0.39 () {60 vars in 1140 bytes} [Mon Aug 8 02:12:24 2022] POST /identity/v3/projects => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:12:24.907783 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 28/55] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:24 2022] GET /identity/v3/users/2d716e3acbb54b2fa8e10b30ba8eb5fb => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:24.913549 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6ec0e-7c70-42ea-bbd6-e5f75d3a0d9c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:25.025408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6ec0e-7c70-42ea-bbd6-e5f75d3a0d9c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:25.062977 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e31c27-cfe4-486f-9556-47459468a18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:25.063770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e31c27-cfe4-486f-9556-47459468a18e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:25.064085 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e31c27-cfe4-486f-9556-47459468a18e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:25.064512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e31c27-cfe4-486f-9556-47459468a18e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:25.065749 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e31c27-cfe4-486f-9556-47459468a18e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:25.067398 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e31c27-cfe4-486f-9556-47459468a18e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:25.069713 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 28/56] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:12:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:25.075665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f435c-f896-40a7-a4c9-4cbd07cd7d55 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:25.118413 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:25.175426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f435c-f896-40a7-a4c9-4cbd07cd7d55 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:25.208924 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b8bae-1e0f-4b80-a407-d451bb2c26e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:25.209556 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b8bae-1e0f-4b80-a407-d451bb2c26e8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:25.209835 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b8bae-1e0f-4b80-a407-d451bb2c26e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:25.210105 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b8bae-1e0f-4b80-a407-d451bb2c26e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:25.210849 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b8bae-1e0f-4b80-a407-d451bb2c26e8 None admin] RBAC: Adding query filter params (role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.domain.id=default, user.id=2d716e3acbb54b2fa8e10b30ba8eb5fb) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:25.211158 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b8bae-1e0f-4b80-a407-d451bb2c26e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:25.212637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b8bae-1e0f-4b80-a407-d451bb2c26e8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:25.219436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 29/57] 10.222.0.39 () {56 vars in 1322 bytes} [Mon Aug 8 02:12:25 2022] GET /identity/v3/role_assignments?user.id=2d716e3acbb54b2fa8e10b30ba8eb5fb&scope.domain.id=default&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:26.923896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016922d4-21c4-4319-a9c2-6d96525bd9e0 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:26.924408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016922d4-21c4-4319-a9c2-6d96525bd9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:26.924517 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016922d4-21c4-4319-a9c2-6d96525bd9e0 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:26.925763 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 29/58] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:12:26.931572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a19040-223d-4af8-9d6d-57dc175684c0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:26.931985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a19040-223d-4af8-9d6d-57dc175684c0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:26.932132 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a19040-223d-4af8-9d6d-57dc175684c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:26.950299 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-58a19040-223d-4af8-9d6d-57dc175684c0 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:26.951497 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-58a19040-223d-4af8-9d6d-57dc175684c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:27.017105 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 30/59] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:27.041752 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016922d4-21c4-4319-a9c2-6d96525bd9e0 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:27.061787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:27.142819 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016922d4-21c4-4319-a9c2-6d96525bd9e0 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:27.175491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e305346-7006-42d2-838a-fd21af197fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:27.176054 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e305346-7006-42d2-838a-fd21af197fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:27.176340 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e305346-7006-42d2-838a-fd21af197fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:27.176618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e305346-7006-42d2-838a-fd21af197fa8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:27.177732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e305346-7006-42d2-838a-fd21af197fa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:27.179254 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e305346-7006-42d2-838a-fd21af197fa8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:27.180802 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 30/60] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:12:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:27.186481 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a19040-223d-4af8-9d6d-57dc175684c0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:27.219212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:27.280247 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a19040-223d-4af8-9d6d-57dc175684c0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:27.312472 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470338b-6f93-48a9-8d97-46bd4f3ce46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:27.313066 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470338b-6f93-48a9-8d97-46bd4f3ce46c None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:27.313344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470338b-6f93-48a9-8d97-46bd4f3ce46c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:27.313614 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470338b-6f93-48a9-8d97-46bd4f3ce46c None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:27.314278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470338b-6f93-48a9-8d97-46bd4f3ce46c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:27.315732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470338b-6f93-48a9-8d97-46bd4f3ce46c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:27.341586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 31/61] 10.222.0.39 () {60 vars in 1132 bytes} [Mon Aug 8 02:12:27 2022] POST /identity/v3/users => generated 275 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:12:30.467283 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7e02e-b1e4-43c1-856f-aeba4c095bfd None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:30.468224 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7e02e-b1e4-43c1-856f-aeba4c095bfd None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:30.468698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7e02e-b1e4-43c1-856f-aeba4c095bfd None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:30.470551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 31/62] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:30.475402 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f33fd-2f3f-4ca9-ab9b-6bc03330ea2d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:30.476191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f33fd-2f3f-4ca9-ab9b-6bc03330ea2d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:30.476677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f33fd-2f3f-4ca9-ab9b-6bc03330ea2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:30.502376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-dc1f33fd-2f3f-4ca9-ab9b-6bc03330ea2d None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:30.503957 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dc1f33fd-2f3f-4ca9-ab9b-6bc03330ea2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:30.548926 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b172be-77c2-4c2b-895b-fc2fb7213a3f None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:30.549235 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b172be-77c2-4c2b-895b-fc2fb7213a3f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:30.549480 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b172be-77c2-4c2b-895b-fc2fb7213a3f None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:30.550836 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 32/63] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:12:30.557663 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17204142-083d-4638-a837-52d0d28c666b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:30.558208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17204142-083d-4638-a837-52d0d28c666b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:30.558739 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17204142-083d-4638-a837-52d0d28c666b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:30.577385 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-17204142-083d-4638-a837-52d0d28c666b None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:30.578933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-17204142-083d-4638-a837-52d0d28c666b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:30.586197 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 32/64] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:30.587580 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d3c38-67c9-4754-960c-eeacae9bb0f3 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:30.587868 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d3c38-67c9-4754-960c-eeacae9bb0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:30.588138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d3c38-67c9-4754-960c-eeacae9bb0f3 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:30.589494 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 33/65] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:30.595845 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c68f6c-23df-4d6d-b5da-70c2fec768a9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:30.596006 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c68f6c-23df-4d6d-b5da-70c2fec768a9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:30.596285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c68f6c-23df-4d6d-b5da-70c2fec768a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:30.624961 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c0c68f6c-23df-4d6d-b5da-70c2fec768a9 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:30.626274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c0c68f6c-23df-4d6d-b5da-70c2fec768a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:30.652869 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:30.655091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 33/66] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:12:30.656219 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17204142-083d-4638-a837-52d0d28c666b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:30.676159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:30.715711 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 34/67] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:30.716479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c68f6c-23df-4d6d-b5da-70c2fec768a9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:30.767379 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17204142-083d-4638-a837-52d0d28c666b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:30.804750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec58560-2102-442d-b9f2-0ee43a27f832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:30.805716 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec58560-2102-442d-b9f2-0ee43a27f832 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:30.806333 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec58560-2102-442d-b9f2-0ee43a27f832 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:30.806807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec58560-2102-442d-b9f2-0ee43a27f832 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:30.818724 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec58560-2102-442d-b9f2-0ee43a27f832 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:30.822844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c68f6c-23df-4d6d-b5da-70c2fec768a9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:30.824195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-aec58560-2102-442d-b9f2-0ee43a27f832 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:30.824195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:30.824195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:30.824195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:30.824195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:30.824195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:30.824195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:30.824195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:30.824195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:30.824195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:30.824195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:30.824195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:30.824195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:30.824195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:30.824195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:30.824195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:30.825047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:30.825047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:30.825047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:30.825047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:30.825047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:30.825047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:30.825047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:30.825047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:30.825047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:30.825047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:30.825047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:30.825047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:30.825047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:30.825047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:30.825047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:30.825047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:30.825798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:30.825798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:30.825798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:30.825798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:30.825798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:30.825798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:30.825798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:30.825798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:30.825798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:30.825798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:30.825798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:30.825798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:12:30.827721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 34/68] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:30.828625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec58560-2102-442d-b9f2-0ee43a27f832 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:30.862177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae9677-ac3a-44ac-b77c-383257005d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:30.862844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae9677-ac3a-44ac-b77c-383257005d30 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:30.863126 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae9677-ac3a-44ac-b77c-383257005d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:30.863400 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae9677-ac3a-44ac-b77c-383257005d30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:30.874188 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae9677-ac3a-44ac-b77c-383257005d30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:30.878956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c1ae9677-ac3a-44ac-b77c-383257005d30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:12:30.878956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:30.878956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:30.878956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:30.878956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:30.878956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:30.878956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:30.878956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:30.878956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:30.878956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:30.878956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:30.878956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:30.878956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:30.878956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:30.878956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:30.878956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:30.879786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:30.879786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:30.879786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:30.879786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:30.879786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:30.879786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:30.879786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:30.879786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:30.879786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:30.879786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:30.879786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:30.879786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:30.879786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:30.879786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:30.879786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:30.879786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:30.880560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:30.880560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:30.880560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:30.880560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:30.880560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:30.880560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:30.880560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:30.880560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:30.880560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:30.880560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:30.880560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:12:30.880560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:30.880560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 35/69] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:12:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:30.881198 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae9677-ac3a-44ac-b77c-383257005d30 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:30.912111 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:30.918600 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:30.939053 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec58560-2102-442d-b9f2-0ee43a27f832 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:30.975766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be009fbf-bc25-44b8-87dc-0f6cdaa3fb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:30.976526 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be009fbf-bc25-44b8-87dc-0f6cdaa3fb06 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:30.976685 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be009fbf-bc25-44b8-87dc-0f6cdaa3fb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:30.977183 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be009fbf-bc25-44b8-87dc-0f6cdaa3fb06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:30.988026 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be009fbf-bc25-44b8-87dc-0f6cdaa3fb06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:30.992855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-be009fbf-bc25-44b8-87dc-0f6cdaa3fb06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:12:30.992855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:30.992855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:30.992855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:30.992855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:30.992855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:30.992855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:30.992855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:30.992855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:30.992855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:30.992855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:30.992855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:30.992855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:30.992855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:30.992855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:30.992855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:30.993676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:30.993676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:30.993676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:30.993676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:30.993676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:30.993676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:30.993676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:30.993676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:30.993676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:30.993676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:30.993676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:30.993676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:30.993676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:30.993676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:30.993676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:30.993676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:30.994459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:30.994459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:30.994459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:30.994459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:30.994459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:30.994459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:30.994459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:30.994459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:30.994459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:30.994459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:30.994459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:12:30.994459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:12:30.994459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 35/70] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:12:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:30.995064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be009fbf-bc25-44b8-87dc-0f6cdaa3fb06 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:31.008773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae9677-ac3a-44ac-b77c-383257005d30 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:31.051302 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a37801-b176-43db-b1c6-f48b93e9ae5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:31.052148 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a37801-b176-43db-b1c6-f48b93e9ae5e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:31.053084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a37801-b176-43db-b1c6-f48b93e9ae5e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:31.053536 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a37801-b176-43db-b1c6-f48b93e9ae5e None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:31.054518 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a37801-b176-43db-b1c6-f48b93e9ae5e None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:31.055010 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a37801-b176-43db-b1c6-f48b93e9ae5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:31.056385 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a37801-b176-43db-b1c6-f48b93e9ae5e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:31.064825 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 36/71] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:12:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:31.065767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a37801-b176-43db-b1c6-f48b93e9ae5e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:31.099415 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be009fbf-bc25-44b8-87dc-0f6cdaa3fb06 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:31.138804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286fbd1-4830-4b07-99ef-83c97903f76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:31.139408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286fbd1-4830-4b07-99ef-83c97903f76f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:31.139685 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286fbd1-4830-4b07-99ef-83c97903f76f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:31.139955 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286fbd1-4830-4b07-99ef-83c97903f76f None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:31.140615 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286fbd1-4830-4b07-99ef-83c97903f76f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:31.140916 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286fbd1-4830-4b07-99ef-83c97903f76f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:31.142323 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286fbd1-4830-4b07-99ef-83c97903f76f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:31.144443 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:31.151886 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 36/72] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:31.153103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5434a-9630-44c7-9329-5fbbbf7fca68 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:31.153392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5434a-9630-44c7-9329-5fbbbf7fca68 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:31.153672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5434a-9630-44c7-9329-5fbbbf7fca68 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:31.155018 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 37/73] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:31.155926 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260073f-57e7-462c-a2d7-9c946350ef69 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:31.156446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260073f-57e7-462c-a2d7-9c946350ef69 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:31.156551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260073f-57e7-462c-a2d7-9c946350ef69 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:31.157705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 38/74] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:31.158444 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260073f-57e7-462c-a2d7-9c946350ef69 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:31.160833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:31.186858 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a37801-b176-43db-b1c6-f48b93e9ae5e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:31.220064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0792d4d-216c-4a16-bc55-b7b64681fef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:31.220668 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0792d4d-216c-4a16-bc55-b7b64681fef4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:31.220957 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0792d4d-216c-4a16-bc55-b7b64681fef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:31.221236 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0792d4d-216c-4a16-bc55-b7b64681fef4 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:31.221899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0792d4d-216c-4a16-bc55-b7b64681fef4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:31.222209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0792d4d-216c-4a16-bc55-b7b64681fef4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:31.223635 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0792d4d-216c-4a16-bc55-b7b64681fef4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:31.232160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 37/75] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:12:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:31.232882 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0792d4d-216c-4a16-bc55-b7b64681fef4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:31.267894 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260073f-57e7-462c-a2d7-9c946350ef69 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:31.301429 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e227ffc-a078-4368-8912-6378c8c3632b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:31.302051 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e227ffc-a078-4368-8912-6378c8c3632b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:31.302369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e227ffc-a078-4368-8912-6378c8c3632b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:31.302648 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e227ffc-a078-4368-8912-6378c8c3632b None admin] PATH_INFO: `/v3/users/907bde105eaf466980f8a555198f3c46` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:31.317401 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e227ffc-a078-4368-8912-6378c8c3632b None admin] RBAC: Authorizing `identity:get_user(user_id=907bde105eaf466980f8a555198f3c46)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:31.319387 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e227ffc-a078-4368-8912-6378c8c3632b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:31.320984 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 39/76] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:31 2022] GET /identity/v3/users/907bde105eaf466980f8a555198f3c46 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:31.322101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864e220-2018-4284-b464-f99916a2fbcf None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:31.322416 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864e220-2018-4284-b464-f99916a2fbcf None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:31.322695 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864e220-2018-4284-b464-f99916a2fbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:31.335002 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0792d4d-216c-4a16-bc55-b7b64681fef4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:31.340084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a864e220-2018-4284-b464-f99916a2fbcf None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:31.341266 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a864e220-2018-4284-b464-f99916a2fbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:31.367368 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21de1d0-8437-440f-9302-25892ca79e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:31.367984 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21de1d0-8437-440f-9302-25892ca79e9d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:31.368264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21de1d0-8437-440f-9302-25892ca79e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:31.368534 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21de1d0-8437-440f-9302-25892ca79e9d None admin] PATH_INFO: `/v3/users/2d716e3acbb54b2fa8e10b30ba8eb5fb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:31.369650 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21de1d0-8437-440f-9302-25892ca79e9d None admin] RBAC: Authorizing `identity:get_user(user_id=2d716e3acbb54b2fa8e10b30ba8eb5fb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:31.371116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21de1d0-8437-440f-9302-25892ca79e9d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:31.372669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 38/77] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:31 2022] GET /identity/v3/users/2d716e3acbb54b2fa8e10b30ba8eb5fb => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:31.377141 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aad14e-5bfb-4cf2-a122-1f93d062b3b1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:31.377141 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aad14e-5bfb-4cf2-a122-1f93d062b3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:31.377141 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aad14e-5bfb-4cf2-a122-1f93d062b3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:31.382564 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:31.391809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:31.392597 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-09aad14e-5bfb-4cf2-a122-1f93d062b3b1 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:31.393663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-09aad14e-5bfb-4cf2-a122-1f93d062b3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:31.409053 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 40/78] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:31.409881 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864e220-2018-4284-b464-f99916a2fbcf None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:31.473348 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 39/79] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:31.474215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aad14e-5bfb-4cf2-a122-1f93d062b3b1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:31.521630 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864e220-2018-4284-b464-f99916a2fbcf None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:31.556053 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f5d2e-47a1-46a7-b860-5b67c26d97ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:31.556676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f5d2e-47a1-46a7-b860-5b67c26d97ce None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:31.556979 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f5d2e-47a1-46a7-b860-5b67c26d97ce None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:31.557247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f5d2e-47a1-46a7-b860-5b67c26d97ce None admin] PATH_INFO: `/v3/users/907bde105eaf466980f8a555198f3c46` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:31.558400 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f5d2e-47a1-46a7-b860-5b67c26d97ce None admin] RBAC: Authorizing `identity:get_user(user_id=907bde105eaf466980f8a555198f3c46)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:31.560101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f5d2e-47a1-46a7-b860-5b67c26d97ce None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:31.561665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 41/80] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:31 2022] GET /identity/v3/users/907bde105eaf466980f8a555198f3c46 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:31.562525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f5d2e-47a1-46a7-b860-5b67c26d97ce None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:31.578226 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aad14e-5bfb-4cf2-a122-1f93d062b3b1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:31.610746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac090c7-41bc-45eb-8aa6-6dba5cc12edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:31.611337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac090c7-41bc-45eb-8aa6-6dba5cc12edd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:31.611616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac090c7-41bc-45eb-8aa6-6dba5cc12edd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:31.611885 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac090c7-41bc-45eb-8aa6-6dba5cc12edd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:31.614414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:31.617987 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:31.662537 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f5d2e-47a1-46a7-b860-5b67c26d97ce None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:31.679159 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac090c7-41bc-45eb-8aa6-6dba5cc12edd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:31.680635 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac090c7-41bc-45eb-8aa6-6dba5cc12edd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:31.697024 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6546fd-c5f1-4b2d-a759-84d0c6aaf3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:31.697629 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6546fd-c5f1-4b2d-a759-84d0c6aaf3ee None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:31.697911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6546fd-c5f1-4b2d-a759-84d0c6aaf3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:31.698181 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6546fd-c5f1-4b2d-a759-84d0c6aaf3ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:31.764434 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6546fd-c5f1-4b2d-a759-84d0c6aaf3ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:31.765902 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6546fd-c5f1-4b2d-a759-84d0c6aaf3ee None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:31.777324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 40/81] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:12:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:12:31.778109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac090c7-41bc-45eb-8aa6-6dba5cc12edd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:31.811878 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:31.820298 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:31.866257 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 42/82] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:12:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:12:31.867110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6546fd-c5f1-4b2d-a759-84d0c6aaf3ee None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:31.875687 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac090c7-41bc-45eb-8aa6-6dba5cc12edd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:31.909210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6dfb6d-e611-4b1d-b34d-cecaae1ef90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:31.909833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dfb6d-e611-4b1d-b34d-cecaae1ef90e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:31.910116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dfb6d-e611-4b1d-b34d-cecaae1ef90e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:31.910429 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dfb6d-e611-4b1d-b34d-cecaae1ef90e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:31.911674 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6dfb6d-e611-4b1d-b34d-cecaae1ef90e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:31.913563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6dfb6d-e611-4b1d-b34d-cecaae1ef90e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:31.915161 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 41/83] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:12:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:31.919119 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6dfb6d-e611-4b1d-b34d-cecaae1ef90e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:31.963787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6546fd-c5f1-4b2d-a759-84d0c6aaf3ee None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:31.997037 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f50ecc-12b8-4678-ba66-900bcc979eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:31.997657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f50ecc-12b8-4678-ba66-900bcc979eab None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:31.997933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f50ecc-12b8-4678-ba66-900bcc979eab None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:31.998207 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f50ecc-12b8-4678-ba66-900bcc979eab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:32.008514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f50ecc-12b8-4678-ba66-900bcc979eab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:32.013195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-54f50ecc-12b8-4678-ba66-900bcc979eab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:32.013195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:32.013195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:32.013195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:32.013195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:32.013195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:32.013195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:32.013195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:32.013195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:32.013195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:32.013195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:32.013195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:32.013195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:32.013195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:32.013195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:32.013195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:32.014001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:32.014001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:32.014001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:32.014001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:32.014001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:32.014001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:32.014001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:32.014001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:32.014001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:32.014001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:32.014001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:32.014001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:32.014001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:32.014001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:32.014001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:32.014001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:32.014776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:32.014776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:32.014776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:32.014776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:32.014776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:32.014776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:32.014776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:32.014776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:32.014776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:32.014776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:32.014776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:32.014776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:12:32.014776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 43/84] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:32.015396 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f50ecc-12b8-4678-ba66-900bcc979eab None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:32.021523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6dfb6d-e611-4b1d-b34d-cecaae1ef90e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:32.023170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:32.030131 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:32.059669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43add4f3-271d-4f86-bb7a-07be7988a13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:32.060268 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43add4f3-271d-4f86-bb7a-07be7988a13c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:32.060551 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43add4f3-271d-4f86-bb7a-07be7988a13c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:32.060824 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43add4f3-271d-4f86-bb7a-07be7988a13c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:32.115242 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f50ecc-12b8-4678-ba66-900bcc979eab None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:32.126671 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43add4f3-271d-4f86-bb7a-07be7988a13c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:32.128142 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43add4f3-271d-4f86-bb7a-07be7988a13c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:32.148782 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaff8d6-5ce4-4707-9aa3-4111fc2261c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:32.149407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaff8d6-5ce4-4707-9aa3-4111fc2261c4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:32.149686 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaff8d6-5ce4-4707-9aa3-4111fc2261c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:32.149958 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaff8d6-5ce4-4707-9aa3-4111fc2261c4 None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:32.151136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaff8d6-5ce4-4707-9aa3-4111fc2261c4 None admin] RBAC: Authorizing `identity:get_project(project_id=3441a04d36074741b13ebfa058e656b8)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:32.152581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaff8d6-5ce4-4707-9aa3-4111fc2261c4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:32.154316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 44/85] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:32 2022] GET /identity/v3/projects/3441a04d36074741b13ebfa058e656b8 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:32.155075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaff8d6-5ce4-4707-9aa3-4111fc2261c4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:32.227190 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 42/86] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:12:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:12:32.228010 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43add4f3-271d-4f86-bb7a-07be7988a13c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:32.230226 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:32.236789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:32.258097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaff8d6-5ce4-4707-9aa3-4111fc2261c4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:32.291165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c920a4-745c-47fb-9fdf-cddc982b1fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:32.291722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c920a4-745c-47fb-9fdf-cddc982b1fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:32.292003 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c920a4-745c-47fb-9fdf-cddc982b1fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:32.292279 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c920a4-745c-47fb-9fdf-cddc982b1fc8 None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:32.293401 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c920a4-745c-47fb-9fdf-cddc982b1fc8 None admin] RBAC: Authorizing `identity:get_project(project_id=3441a04d36074741b13ebfa058e656b8)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:32.294888 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c920a4-745c-47fb-9fdf-cddc982b1fc8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:32.296710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 45/87] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:32 2022] GET /identity/v3/projects/3441a04d36074741b13ebfa058e656b8 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:32.297262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c920a4-745c-47fb-9fdf-cddc982b1fc8 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:32.331265 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43add4f3-271d-4f86-bb7a-07be7988a13c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:32.364154 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566036c2-8b31-450d-a27f-3354b7e2629a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:32.364757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566036c2-8b31-450d-a27f-3354b7e2629a None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:32.365040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566036c2-8b31-450d-a27f-3354b7e2629a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:32.365314 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566036c2-8b31-450d-a27f-3354b7e2629a None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:32.366531 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566036c2-8b31-450d-a27f-3354b7e2629a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:32.367980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566036c2-8b31-450d-a27f-3354b7e2629a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:32.388591 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 43/88] 10.222.0.39 () {60 vars in 1132 bytes} [Mon Aug 8 02:12:32 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:12:32.389408 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566036c2-8b31-450d-a27f-3354b7e2629a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:32.401476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c920a4-745c-47fb-9fdf-cddc982b1fc8 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:32.448098 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae69e8-ebda-467d-a057-8405ff3da237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:32.448728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae69e8-ebda-467d-a057-8405ff3da237 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:32.449007 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae69e8-ebda-467d-a057-8405ff3da237 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:32.449277 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae69e8-ebda-467d-a057-8405ff3da237 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:32.449952 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae69e8-ebda-467d-a057-8405ff3da237 None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:32.450280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae69e8-ebda-467d-a057-8405ff3da237 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:32.451670 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae69e8-ebda-467d-a057-8405ff3da237 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:32.461222 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 46/89] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:12:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:32.462099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae69e8-ebda-467d-a057-8405ff3da237 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:32.480654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:32.483216 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:32.535548 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566036c2-8b31-450d-a27f-3354b7e2629a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:32.576818 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e8493-99e3-4f8a-a43a-afd4c263ff88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:32.577360 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e8493-99e3-4f8a-a43a-afd4c263ff88 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:32.578837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e8493-99e3-4f8a-a43a-afd4c263ff88 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:32.578837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e8493-99e3-4f8a-a43a-afd4c263ff88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:32.578837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e8493-99e3-4f8a-a43a-afd4c263ff88 None admin] RBAC: Adding query filter params (role.id=42c846f6944b40c8af5eeb8360cb7c7a, scope.project.id=3441a04d36074741b13ebfa058e656b8, user.id=907bde105eaf466980f8a555198f3c46) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:32.579451 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e8493-99e3-4f8a-a43a-afd4c263ff88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:32.580414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e8493-99e3-4f8a-a43a-afd4c263ff88 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:32.586734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 44/90] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:12:32 2022] GET /identity/v3/role_assignments?user.id=907bde105eaf466980f8a555198f3c46&scope.project.id=3441a04d36074741b13ebfa058e656b8&role.id=42c846f6944b40c8af5eeb8360cb7c7a => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:32.587509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e8493-99e3-4f8a-a43a-afd4c263ff88 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:32.599701 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae69e8-ebda-467d-a057-8405ff3da237 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:32.657200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df26b09-c503-4395-b07b-b1d590a84143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:32.657864 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df26b09-c503-4395-b07b-b1d590a84143 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:32.658187 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df26b09-c503-4395-b07b-b1d590a84143 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:32.658461 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df26b09-c503-4395-b07b-b1d590a84143 None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:32.660906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df26b09-c503-4395-b07b-b1d590a84143 None admin] RBAC: Authorizing `identity:get_project(project_id=3441a04d36074741b13ebfa058e656b8)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:32.663264 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df26b09-c503-4395-b07b-b1d590a84143 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:32.665416 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 47/91] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:32 2022] GET /identity/v3/projects/3441a04d36074741b13ebfa058e656b8 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:32.666628 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df26b09-c503-4395-b07b-b1d590a84143 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:32.716009 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e8493-99e3-4f8a-a43a-afd4c263ff88 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:32.746651 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:32.760444 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226a048-7d56-4802-8717-5179dd893860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:32.761828 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226a048-7d56-4802-8717-5179dd893860 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:32.762829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226a048-7d56-4802-8717-5179dd893860 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:32.762829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226a048-7d56-4802-8717-5179dd893860 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:32.763147 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226a048-7d56-4802-8717-5179dd893860 None admin] RBAC: Adding query filter params (role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.project.id=3441a04d36074741b13ebfa058e656b8, user.id=2d716e3acbb54b2fa8e10b30ba8eb5fb) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:32.763462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226a048-7d56-4802-8717-5179dd893860 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:32.764978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226a048-7d56-4802-8717-5179dd893860 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:32.766709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:32.776010 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 45/92] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:12:32 2022] GET /identity/v3/role_assignments?user.id=2d716e3acbb54b2fa8e10b30ba8eb5fb&scope.project.id=3441a04d36074741b13ebfa058e656b8&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:32.776010 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226a048-7d56-4802-8717-5179dd893860 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:32.809312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df26b09-c503-4395-b07b-b1d590a84143 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:32.891193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516f9ea-7525-4129-91f9-1562b112efa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:32.891764 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516f9ea-7525-4129-91f9-1562b112efa1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:32.892044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516f9ea-7525-4129-91f9-1562b112efa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:32.892319 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516f9ea-7525-4129-91f9-1562b112efa1 None admin] PATH_INFO: `/v3/users/907bde105eaf466980f8a555198f3c46` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:32.893612 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516f9ea-7525-4129-91f9-1562b112efa1 None admin] RBAC: Authorizing `identity:get_user(user_id=907bde105eaf466980f8a555198f3c46)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:32.895112 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516f9ea-7525-4129-91f9-1562b112efa1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:32.897627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 48/93] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:32 2022] GET /identity/v3/users/907bde105eaf466980f8a555198f3c46 => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:32.902818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516f9ea-7525-4129-91f9-1562b112efa1 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:32.981175 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226a048-7d56-4802-8717-5179dd893860 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:33.029272 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75ef17-07d1-44a7-876d-66f5aac0006d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:33.030519 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75ef17-07d1-44a7-876d-66f5aac0006d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:33.030995 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75ef17-07d1-44a7-876d-66f5aac0006d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:33.031454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75ef17-07d1-44a7-876d-66f5aac0006d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:33.032760 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75ef17-07d1-44a7-876d-66f5aac0006d None admin] RBAC: Adding query filter params (role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839, scope.project.id=3441a04d36074741b13ebfa058e656b8, user.id=907bde105eaf466980f8a555198f3c46) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:33.033384 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75ef17-07d1-44a7-876d-66f5aac0006d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:33.035618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75ef17-07d1-44a7-876d-66f5aac0006d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:33.042508 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 46/94] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:12:32 2022] GET /identity/v3/role_assignments?user.id=907bde105eaf466980f8a555198f3c46&scope.project.id=3441a04d36074741b13ebfa058e656b8&role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839 => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:33.067287 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516f9ea-7525-4129-91f9-1562b112efa1 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:33.122105 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d98892-28b2-43c5-8e86-7ea10531947d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:33.122751 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d98892-28b2-43c5-8e86-7ea10531947d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:33.123032 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d98892-28b2-43c5-8e86-7ea10531947d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:33.123305 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d98892-28b2-43c5-8e86-7ea10531947d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:33.155138 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:33.224658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d98892-28b2-43c5-8e86-7ea10531947d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:33.226404 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d98892-28b2-43c5-8e86-7ea10531947d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:33.370540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 49/95] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:12:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 02:12:33.376170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75ef17-07d1-44a7-876d-66f5aac0006d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:33.463386 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:33.517966 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75ef17-07d1-44a7-876d-66f5aac0006d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:33.552895 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2609e2c-ad49-4c32-99bf-7d9bd6ca25de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:33.554071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2609e2c-ad49-4c32-99bf-7d9bd6ca25de None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:33.554378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2609e2c-ad49-4c32-99bf-7d9bd6ca25de None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:33.554647 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2609e2c-ad49-4c32-99bf-7d9bd6ca25de None admin] PATH_INFO: `/v3/projects/27caf57dbf92480283ca4dd69f9d0a43` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:33.555932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2609e2c-ad49-4c32-99bf-7d9bd6ca25de None admin] RBAC: Authorizing `identity:get_project(project_id=27caf57dbf92480283ca4dd69f9d0a43)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:33.557453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2609e2c-ad49-4c32-99bf-7d9bd6ca25de None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:33.559177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 47/96] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:33 2022] GET /identity/v3/projects/27caf57dbf92480283ca4dd69f9d0a43 => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:33.571375 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d98892-28b2-43c5-8e86-7ea10531947d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:33.648081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:33.713806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d98892-28b2-43c5-8e86-7ea10531947d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:33.759488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085dcf4-1f45-46bf-a037-5a1e7d6d3bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:33.760117 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085dcf4-1f45-46bf-a037-5a1e7d6d3bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:33.760401 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085dcf4-1f45-46bf-a037-5a1e7d6d3bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:33.760680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085dcf4-1f45-46bf-a037-5a1e7d6d3bf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:33.761416 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085dcf4-1f45-46bf-a037-5a1e7d6d3bf3 None admin] RBAC: Adding query filter params (role.id=42c846f6944b40c8af5eeb8360cb7c7a, scope.project.id=27caf57dbf92480283ca4dd69f9d0a43, user.id=907bde105eaf466980f8a555198f3c46) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:33.761732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085dcf4-1f45-46bf-a037-5a1e7d6d3bf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:33.763199 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085dcf4-1f45-46bf-a037-5a1e7d6d3bf3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:33.769074 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 50/97] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:12:33 2022] GET /identity/v3/role_assignments?user.id=907bde105eaf466980f8a555198f3c46&scope.project.id=27caf57dbf92480283ca4dd69f9d0a43&role.id=42c846f6944b40c8af5eeb8360cb7c7a => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:35.359804 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795297c9-8f61-4a0a-9c4c-bf16face4311 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:35.360310 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795297c9-8f61-4a0a-9c4c-bf16face4311 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:35.360429 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795297c9-8f61-4a0a-9c4c-bf16face4311 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:35.361816 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 48/98] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:35.381076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c383f3-4b3c-4cc0-8009-7863e6b36241 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:35.381980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c383f3-4b3c-4cc0-8009-7863e6b36241 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:35.382487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c383f3-4b3c-4cc0-8009-7863e6b36241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:35.411254 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b4c383f3-4b3c-4cc0-8009-7863e6b36241 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:35.413365 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b4c383f3-4b3c-4cc0-8009-7863e6b36241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:35.496803 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 51/99] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:35.535555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795297c9-8f61-4a0a-9c4c-bf16face4311 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:35.564215 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513772a7-0151-4ea4-af32-f7b0c4a885b3 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:35.564819 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513772a7-0151-4ea4-af32-f7b0c4a885b3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:35.565285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513772a7-0151-4ea4-af32-f7b0c4a885b3 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:35.566843 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 52/100] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:12:35.572057 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b648b4-477b-485f-9413-b5594f6d1e7f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:35.572580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b648b4-477b-485f-9413-b5594f6d1e7f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:35.573100 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b648b4-477b-485f-9413-b5594f6d1e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:35.593882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-07b648b4-477b-485f-9413-b5594f6d1e7f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:35.595416 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-07b648b4-477b-485f-9413-b5594f6d1e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:35.671428 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:35.671883 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795297c9-8f61-4a0a-9c4c-bf16face4311 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:35.707450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:35.728829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 53/101] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:12:35.741733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff6d7f-ad70-4980-8619-c4e492220fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:35.742398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff6d7f-ad70-4980-8619-c4e492220fbd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:35.742680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff6d7f-ad70-4980-8619-c4e492220fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:35.742955 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff6d7f-ad70-4980-8619-c4e492220fbd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:35.753963 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff6d7f-ad70-4980-8619-c4e492220fbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:35.758781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-abff6d7f-ad70-4980-8619-c4e492220fbd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:12:35.758781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:35.758781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:35.758781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:35.758781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:35.758781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:35.758781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:35.758781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:35.758781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:35.758781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:35.758781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:35.758781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:35.758781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:35.758781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:35.758781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:35.758781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:35.759604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:35.759604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:35.759604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:35.759604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:35.759604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:35.759604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:35.759604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:35.759604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:35.759604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:35.759604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:35.759604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:35.759604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:35.759604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:35.759604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:35.759604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:35.759604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:35.760331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:35.760331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:35.760331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:35.760331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:35.760331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:35.760331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:35.760331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:35.760331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:35.760331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:35.760331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:35.760331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:12:35.760331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:35.760331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 49/102] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:35.766952 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b648b4-477b-485f-9413-b5594f6d1e7f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:35.777042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff6d7f-ad70-4980-8619-c4e492220fbd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:35.895635 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b648b4-477b-485f-9413-b5594f6d1e7f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:35.916738 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff6d7f-ad70-4980-8619-c4e492220fbd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:35.934215 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b84f19-8f69-4f60-ba83-759bbc82dd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:35.935425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b84f19-8f69-4f60-ba83-759bbc82dd2e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:35.935967 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b84f19-8f69-4f60-ba83-759bbc82dd2e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:35.936408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b84f19-8f69-4f60-ba83-759bbc82dd2e None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:35.937310 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b84f19-8f69-4f60-ba83-759bbc82dd2e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:35.937792 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b84f19-8f69-4f60-ba83-759bbc82dd2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:35.939504 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b84f19-8f69-4f60-ba83-759bbc82dd2e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:35.950375 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 54/103] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:12:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:35.952111 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ac33f-a671-4e32-a1bc-f87da375fab7 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:35.952551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ac33f-a671-4e32-a1bc-f87da375fab7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:35.952986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ac33f-a671-4e32-a1bc-f87da375fab7 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:35.958299 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 55/104] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:35.958299 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9463571-f669-496a-9bbe-2ced29714c70 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:35.958299 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9463571-f669-496a-9bbe-2ced29714c70 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:35.958527 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9463571-f669-496a-9bbe-2ced29714c70 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:35.961348 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 56/105] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:35.961455 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd7809-867d-469c-8c71-408ef5092f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:35.961565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9463571-f669-496a-9bbe-2ced29714c70 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:35.961634 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd7809-867d-469c-8c71-408ef5092f13 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:35.961969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd7809-867d-469c-8c71-408ef5092f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:35.962224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd7809-867d-469c-8c71-408ef5092f13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:35.973205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd7809-867d-469c-8c71-408ef5092f13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:35.978367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-29dd7809-867d-469c-8c71-408ef5092f13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:12:35.978367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:35.978367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:35.978367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:35.978367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:35.978367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:35.978367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:35.978367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:35.978367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:35.978367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:35.978367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:35.978367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:35.978367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:35.978367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:35.978367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:35.978367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:35.979169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:35.979169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:35.979169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:35.979169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:35.979169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:35.979169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:35.979169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:35.979169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:35.979169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:35.979169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:35.979169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:35.979169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:35.979169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:35.979169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:35.979169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:35.979169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:35.979922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:35.979922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:35.979922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:35.979922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:35.979922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:35.979922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:35.979922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:35.979922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:35.979922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:35.979922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:35.979922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:12:35.979922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:35.979922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 50/106] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:12:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:35.981176 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c74e52-df97-4f9b-a0d3-9d173d05acef None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:35.981176 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c74e52-df97-4f9b-a0d3-9d173d05acef None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:35.981176 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c74e52-df97-4f9b-a0d3-9d173d05acef None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:35.982508 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 51/107] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:35.983325 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21aafc-3e4e-4179-8db5-2d35cba207fd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:35.983601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21aafc-3e4e-4179-8db5-2d35cba207fd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:35.983865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21aafc-3e4e-4179-8db5-2d35cba207fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:36.002042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ea21aafc-3e4e-4179-8db5-2d35cba207fd None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:36.003250 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ea21aafc-3e4e-4179-8db5-2d35cba207fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:36.007936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:36.035733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:36.076799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 52/108] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:36.083470 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9463571-f669-496a-9bbe-2ced29714c70 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:36.085775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f5212-1fd1-4976-84b0-004406f11da4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:36.086129 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f5212-1fd1-4976-84b0-004406f11da4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:36.086497 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f5212-1fd1-4976-84b0-004406f11da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:36.104211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0b4f5212-1fd1-4976-84b0-004406f11da4 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:36.105436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0b4f5212-1fd1-4976-84b0-004406f11da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:36.127283 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c517b-dc51-4b9e-9097-7ef0a70ffb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:36.128146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c517b-dc51-4b9e-9097-7ef0a70ffb82 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:36.128507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c517b-dc51-4b9e-9097-7ef0a70ffb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:36.128785 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c517b-dc51-4b9e-9097-7ef0a70ffb82 None admin] PATH_INFO: `/v3/users/8ef3ff73ef554c22aa1dec9bab091558` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:36.142406 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c517b-dc51-4b9e-9097-7ef0a70ffb82 None admin] RBAC: Authorizing `identity:get_user(user_id=8ef3ff73ef554c22aa1dec9bab091558)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:36.143853 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c517b-dc51-4b9e-9097-7ef0a70ffb82 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:36.150353 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 57/109] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:35 2022] GET /identity/v3/users/8ef3ff73ef554c22aa1dec9bab091558 => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:36.150353 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c517b-dc51-4b9e-9097-7ef0a70ffb82 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:36.174617 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 53/110] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:36.185591 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d84574-be08-45da-9a06-d18903b6037c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:36.185865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d84574-be08-45da-9a06-d18903b6037c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:36.186107 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d84574-be08-45da-9a06-d18903b6037c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:36.206233 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a7d84574-be08-45da-9a06-d18903b6037c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:36.207527 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a7d84574-be08-45da-9a06-d18903b6037c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:36.248495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:36.261945 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c517b-dc51-4b9e-9097-7ef0a70ffb82 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:36.276695 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 54/111] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:36.277442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d84574-be08-45da-9a06-d18903b6037c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:36.306498 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604d78a-3eb8-48cf-b04f-2db9ecdd4618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:36.307101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604d78a-3eb8-48cf-b04f-2db9ecdd4618 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:36.307391 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604d78a-3eb8-48cf-b04f-2db9ecdd4618 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:36.307649 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604d78a-3eb8-48cf-b04f-2db9ecdd4618 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:36.308311 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604d78a-3eb8-48cf-b04f-2db9ecdd4618 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:36.308616 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604d78a-3eb8-48cf-b04f-2db9ecdd4618 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:36.310383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604d78a-3eb8-48cf-b04f-2db9ecdd4618 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:36.315870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:36.325002 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 58/112] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:36.325002 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604d78a-3eb8-48cf-b04f-2db9ecdd4618 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:36.399553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d84574-be08-45da-9a06-d18903b6037c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:36.434044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604d78a-3eb8-48cf-b04f-2db9ecdd4618 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:36.438743 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd67e2-3629-4616-a881-15c5ee03ecda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:36.439316 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd67e2-3629-4616-a881-15c5ee03ecda None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:36.439591 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd67e2-3629-4616-a881-15c5ee03ecda None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:36.439859 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd67e2-3629-4616-a881-15c5ee03ecda None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:36.443202 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd67e2-3629-4616-a881-15c5ee03ecda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:36.445418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd67e2-3629-4616-a881-15c5ee03ecda None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:36.452107 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 55/113] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:12:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:36.452107 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd67e2-3629-4616-a881-15c5ee03ecda None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:36.475810 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ed6ae-9f64-4a6e-b2c3-ebf483ff2151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:36.476419 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ed6ae-9f64-4a6e-b2c3-ebf483ff2151 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:36.476697 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ed6ae-9f64-4a6e-b2c3-ebf483ff2151 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:36.476969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ed6ae-9f64-4a6e-b2c3-ebf483ff2151 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:36.502379 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:36.547706 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ed6ae-9f64-4a6e-b2c3-ebf483ff2151 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:36.549217 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ed6ae-9f64-4a6e-b2c3-ebf483ff2151 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:36.554036 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:36.561838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd67e2-3629-4616-a881-15c5ee03ecda None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:36.596615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c75b5-d1a0-4943-b033-086873215c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:36.597370 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c75b5-d1a0-4943-b033-086873215c5e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:36.597645 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c75b5-d1a0-4943-b033-086873215c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:36.597912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c75b5-d1a0-4943-b033-086873215c5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:36.609777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c75b5-d1a0-4943-b033-086873215c5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:36.614736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-042c75b5-d1a0-4943-b033-086873215c5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:36.614736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:36.614736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:36.614736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:36.614736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:36.614736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:36.614736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:36.614736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:36.614736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:36.614736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:36.614736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:36.614736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:36.614736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:36.614736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:36.614736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:36.614736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:36.616145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:36.616145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:36.616145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:36.616145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:36.616145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:36.616145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:36.616145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:36.616145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:36.616145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:36.616145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:36.616145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:36.616145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:36.616145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:36.616145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:36.616145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:36.616145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:36.616899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:36.616899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:36.616899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:36.616899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:36.616899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:36.616899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:36.616899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:36.616899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:36.616899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:36.616899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:36.616899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:36.616899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:36.621361 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 56/114] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:36.622202 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c75b5-d1a0-4943-b033-086873215c5e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:36.649233 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 59/115] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:12:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:12:36.654501 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ed6ae-9f64-4a6e-b2c3-ebf483ff2151 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:36.709553 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:36.727643 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c75b5-d1a0-4943-b033-086873215c5e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:36.755523 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ed6ae-9f64-4a6e-b2c3-ebf483ff2151 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:36.762854 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458a7b7-731a-42ce-9778-f26e80efb313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:36.763587 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458a7b7-731a-42ce-9778-f26e80efb313 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:36.763865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458a7b7-731a-42ce-9778-f26e80efb313 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:36.764132 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458a7b7-731a-42ce-9778-f26e80efb313 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:36.774876 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458a7b7-731a-42ce-9778-f26e80efb313 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:36.779910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f458a7b7-731a-42ce-9778-f26e80efb313 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:12:36.779910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:36.779910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:36.779910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:36.779910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:36.779910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:36.779910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:36.779910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:36.779910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:36.779910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:36.779910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:36.779910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:36.779910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:36.779910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:36.779910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:36.779910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:36.780757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:36.780757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:36.780757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:36.780757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:36.780757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:36.780757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:36.780757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:36.780757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:36.780757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:36.780757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:36.780757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:36.780757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:36.780757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:36.780757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:36.780757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:36.780757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:36.786703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:36.786703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:36.786703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:36.786703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:36.786703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:36.786703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:36.786703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:36.786703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:36.786703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:36.786703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:36.786703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:12:36.786703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:36.786703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 57/116] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:12:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:36.786703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458a7b7-731a-42ce-9778-f26e80efb313 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:36.796809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95baed-a65e-4aa0-b9c2-e2831bed47f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:36.797446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95baed-a65e-4aa0-b9c2-e2831bed47f5 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:36.797728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95baed-a65e-4aa0-b9c2-e2831bed47f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:36.798001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95baed-a65e-4aa0-b9c2-e2831bed47f5 None admin] PATH_INFO: `/v3/users/2d716e3acbb54b2fa8e10b30ba8eb5fb` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:36.799146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95baed-a65e-4aa0-b9c2-e2831bed47f5 None admin] RBAC: Authorizing `identity:get_user(user_id=2d716e3acbb54b2fa8e10b30ba8eb5fb)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:36.800599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95baed-a65e-4aa0-b9c2-e2831bed47f5 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:36.802385 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 60/117] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:36 2022] GET /identity/v3/users/2d716e3acbb54b2fa8e10b30ba8eb5fb => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:36.803595 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95baed-a65e-4aa0-b9c2-e2831bed47f5 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:36.810674 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:36.891486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458a7b7-731a-42ce-9778-f26e80efb313 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:36.909065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95baed-a65e-4aa0-b9c2-e2831bed47f5 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:36.924616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699477b0-2506-4b5b-885d-6a12d116c626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:36.925212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699477b0-2506-4b5b-885d-6a12d116c626 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:36.925489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699477b0-2506-4b5b-885d-6a12d116c626 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:36.925763 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699477b0-2506-4b5b-885d-6a12d116c626 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:36.926429 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699477b0-2506-4b5b-885d-6a12d116c626 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:36.927897 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699477b0-2506-4b5b-885d-6a12d116c626 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:36.932025 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:36.943720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73437093-7e94-46fd-adfa-d1a32ee7b02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:36.944341 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73437093-7e94-46fd-adfa-d1a32ee7b02f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:36.944623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73437093-7e94-46fd-adfa-d1a32ee7b02f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:36.944896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73437093-7e94-46fd-adfa-d1a32ee7b02f None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:36.945631 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73437093-7e94-46fd-adfa-d1a32ee7b02f None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:36.945969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73437093-7e94-46fd-adfa-d1a32ee7b02f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:36.947362 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73437093-7e94-46fd-adfa-d1a32ee7b02f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:36.954976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 58/118] 10.222.0.39 () {60 vars in 1141 bytes} [Mon Aug 8 02:12:36 2022] POST /identity/v3/projects => generated 302 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:12:36.961047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 61/119] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:12:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:36.961179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699477b0-2506-4b5b-885d-6a12d116c626 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:36.961265 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73437093-7e94-46fd-adfa-d1a32ee7b02f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:37.039294 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:37.081361 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73437093-7e94-46fd-adfa-d1a32ee7b02f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:37.082206 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699477b0-2506-4b5b-885d-6a12d116c626 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:37.121897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b246c-8751-432c-a18a-93f2d3366c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:37.122129 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b246c-8751-432c-a18a-93f2d3366c77 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:37.122382 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b246c-8751-432c-a18a-93f2d3366c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:37.122648 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b246c-8751-432c-a18a-93f2d3366c77 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:37.123030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6c8e6-b470-4299-9db2-24f20b931bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:37.123356 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b246c-8751-432c-a18a-93f2d3366c77 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:37.123672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b246c-8751-432c-a18a-93f2d3366c77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:37.123761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6c8e6-b470-4299-9db2-24f20b931bdf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:37.123950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6c8e6-b470-4299-9db2-24f20b931bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:37.124224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6c8e6-b470-4299-9db2-24f20b931bdf None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:37.125067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b246c-8751-432c-a18a-93f2d3366c77 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:37.126086 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6c8e6-b470-4299-9db2-24f20b931bdf None admin] RBAC: Authorizing `identity:get_project(project_id=3441a04d36074741b13ebfa058e656b8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:37.127432 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6c8e6-b470-4299-9db2-24f20b931bdf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:37.135605 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 59/120] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:36 2022] GET /identity/v3/projects/3441a04d36074741b13ebfa058e656b8 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:37.135605 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6c8e6-b470-4299-9db2-24f20b931bdf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:37.140263 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 62/121] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:12:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:37.141565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29550313-6285-406e-8f69-93ffae66cc5f None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:37.142549 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29550313-6285-406e-8f69-93ffae66cc5f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:37.142849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29550313-6285-406e-8f69-93ffae66cc5f None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:37.144234 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 63/122] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:37.145003 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29550313-6285-406e-8f69-93ffae66cc5f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:37.202685 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:37.244526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6c8e6-b470-4299-9db2-24f20b931bdf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:37.262716 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29550313-6285-406e-8f69-93ffae66cc5f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:37.277509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127cd09e-fdc8-41f5-bfff-2961f4ff07fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:37.278134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cd09e-fdc8-41f5-bfff-2961f4ff07fa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:37.278444 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cd09e-fdc8-41f5-bfff-2961f4ff07fa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:37.278715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cd09e-fdc8-41f5-bfff-2961f4ff07fa None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:37.279815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127cd09e-fdc8-41f5-bfff-2961f4ff07fa None admin] RBAC: Authorizing `identity:get_project(project_id=3441a04d36074741b13ebfa058e656b8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:37.281266 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127cd09e-fdc8-41f5-bfff-2961f4ff07fa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:37.282906 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 60/123] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:37 2022] GET /identity/v3/projects/3441a04d36074741b13ebfa058e656b8 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:37.283664 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127cd09e-fdc8-41f5-bfff-2961f4ff07fa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:37.291858 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:37.302561 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38940e0-8845-4fe7-b540-63e026d79c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:37.303428 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38940e0-8845-4fe7-b540-63e026d79c01 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:37.303857 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38940e0-8845-4fe7-b540-63e026d79c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:37.304310 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38940e0-8845-4fe7-b540-63e026d79c01 None admin] PATH_INFO: `/v3/users/907bde105eaf466980f8a555198f3c46` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:37.305604 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38940e0-8845-4fe7-b540-63e026d79c01 None admin] RBAC: Authorizing `identity:get_user(user_id=907bde105eaf466980f8a555198f3c46)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:37.307227 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38940e0-8845-4fe7-b540-63e026d79c01 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:37.308961 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 64/124] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:37 2022] GET /identity/v3/users/907bde105eaf466980f8a555198f3c46 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:37.310111 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38940e0-8845-4fe7-b540-63e026d79c01 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:37.400801 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127cd09e-fdc8-41f5-bfff-2961f4ff07fa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:37.423359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38940e0-8845-4fe7-b540-63e026d79c01 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:37.441321 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:37.446227 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3e938-eb5a-403b-ab36-e2af58a36e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:37.446944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3e938-eb5a-403b-ab36-e2af58a36e5a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:37.447238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3e938-eb5a-403b-ab36-e2af58a36e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:37.447515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3e938-eb5a-403b-ab36-e2af58a36e5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:37.448676 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3e938-eb5a-403b-ab36-e2af58a36e5a None admin] RBAC: Adding query filter params (role.id=3b1d23e3e45d44a3a8069b8e823410b5, scope.project.id=3441a04d36074741b13ebfa058e656b8, user.id=8ef3ff73ef554c22aa1dec9bab091558) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:37.448676 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3e938-eb5a-403b-ab36-e2af58a36e5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:37.450014 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3e938-eb5a-403b-ab36-e2af58a36e5a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:37.464681 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 61/125] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:12:37 2022] GET /identity/v3/role_assignments?user.id=8ef3ff73ef554c22aa1dec9bab091558&scope.project.id=3441a04d36074741b13ebfa058e656b8&role.id=3b1d23e3e45d44a3a8069b8e823410b5 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:37.464681 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b6679-c714-4a13-b0e8-6546bd03e7aa None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:37.464681 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b6679-c714-4a13-b0e8-6546bd03e7aa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:37.464681 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b6679-c714-4a13-b0e8-6546bd03e7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:37.474839 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df3994-4aaa-48af-904e-4ff07857a446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:37.475600 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df3994-4aaa-48af-904e-4ff07857a446 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:37.475948 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df3994-4aaa-48af-904e-4ff07857a446 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:37.476287 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df3994-4aaa-48af-904e-4ff07857a446 None admin] PATH_INFO: `/v3/users/907bde105eaf466980f8a555198f3c46` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:37.477735 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df3994-4aaa-48af-904e-4ff07857a446 None admin] RBAC: Authorizing `identity:get_user(user_id=907bde105eaf466980f8a555198f3c46)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:37.478495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d85b6679-c714-4a13-b0e8-6546bd03e7aa None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:37.479398 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df3994-4aaa-48af-904e-4ff07857a446 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:37.479753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d85b6679-c714-4a13-b0e8-6546bd03e7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:37.481389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 65/126] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:37 2022] GET /identity/v3/users/907bde105eaf466980f8a555198f3c46 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:37.482150 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df3994-4aaa-48af-904e-4ff07857a446 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:37.565492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:37.572613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 62/127] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:37.573902 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b6679-c714-4a13-b0e8-6546bd03e7aa None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:37.616376 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df3994-4aaa-48af-904e-4ff07857a446 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:37.663099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4374a2-c323-445c-88f8-0d9f02c6fbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:37.663875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4374a2-c323-445c-88f8-0d9f02c6fbcf None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:37.664224 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4374a2-c323-445c-88f8-0d9f02c6fbcf None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:37.664573 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4374a2-c323-445c-88f8-0d9f02c6fbcf None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8/users/2d716e3acbb54b2fa8e10b30ba8eb5fb/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:37.667266 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4374a2-c323-445c-88f8-0d9f02c6fbcf None admin] RBAC: Authorizing `identity:create_grant(project_id=3441a04d36074741b13ebfa058e656b8, user_id=2d716e3acbb54b2fa8e10b30ba8eb5fb, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:37.669342 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4374a2-c323-445c-88f8-0d9f02c6fbcf None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:37.692975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 66/128] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:12:37 2022] PUT /identity/v3/projects/3441a04d36074741b13ebfa058e656b8/users/2d716e3acbb54b2fa8e10b30ba8eb5fb/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:12:37.698316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4374a2-c323-445c-88f8-0d9f02c6fbcf None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:37.700235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b6679-c714-4a13-b0e8-6546bd03e7aa None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:37.751386 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:37.775083 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583403e7-6527-4dd2-98db-3af4c6dd61ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:37.775752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583403e7-6527-4dd2-98db-3af4c6dd61ac None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:37.776035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583403e7-6527-4dd2-98db-3af4c6dd61ac None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:37.776307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583403e7-6527-4dd2-98db-3af4c6dd61ac None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:37.777872 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583403e7-6527-4dd2-98db-3af4c6dd61ac None admin] RBAC: Authorizing `identity:get_project(project_id=3441a04d36074741b13ebfa058e656b8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:37.779417 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583403e7-6527-4dd2-98db-3af4c6dd61ac None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:37.781346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 63/129] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:37 2022] GET /identity/v3/projects/3441a04d36074741b13ebfa058e656b8 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:37.781974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583403e7-6527-4dd2-98db-3af4c6dd61ac None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:37.827822 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4374a2-c323-445c-88f8-0d9f02c6fbcf None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:37.872492 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da5f18-786c-4266-bcf4-382a5d0a6e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:37.873613 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da5f18-786c-4266-bcf4-382a5d0a6e43 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:37.873899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da5f18-786c-4266-bcf4-382a5d0a6e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:37.874177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da5f18-786c-4266-bcf4-382a5d0a6e43 None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:37.875416 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da5f18-786c-4266-bcf4-382a5d0a6e43 None admin] RBAC: Authorizing `identity:get_project(project_id=3441a04d36074741b13ebfa058e656b8)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:37.877281 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da5f18-786c-4266-bcf4-382a5d0a6e43 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:37.879697 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 67/130] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:37 2022] GET /identity/v3/projects/3441a04d36074741b13ebfa058e656b8 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:37.880563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da5f18-786c-4266-bcf4-382a5d0a6e43 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:37.934237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:37.974077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583403e7-6527-4dd2-98db-3af4c6dd61ac None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:37.988626 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da5f18-786c-4266-bcf4-382a5d0a6e43 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:37.999823 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:38.014479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffd013-ef65-40b2-90b5-a414b7940538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:38.015442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffd013-ef65-40b2-90b5-a414b7940538 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:38.015911 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffd013-ef65-40b2-90b5-a414b7940538 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:38.016479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffd013-ef65-40b2-90b5-a414b7940538 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:38.025361 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362dd8a-00e3-473a-b1d5-ef2a49dcfe32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:38.026020 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362dd8a-00e3-473a-b1d5-ef2a49dcfe32 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:38.026329 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362dd8a-00e3-473a-b1d5-ef2a49dcfe32 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:38.026609 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362dd8a-00e3-473a-b1d5-ef2a49dcfe32 None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8/users/907bde105eaf466980f8a555198f3c46/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:38.028552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362dd8a-00e3-473a-b1d5-ef2a49dcfe32 None admin] RBAC: Authorizing `identity:create_grant(project_id=3441a04d36074741b13ebfa058e656b8, user_id=907bde105eaf466980f8a555198f3c46, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:38.028970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffd013-ef65-40b2-90b5-a414b7940538 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:38.029997 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362dd8a-00e3-473a-b1d5-ef2a49dcfe32 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:38.035171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-deffd013-ef65-40b2-90b5-a414b7940538 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:38.035171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:38.035171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:38.035171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:38.035171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:38.035171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:38.035171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:38.035171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:38.035171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:38.035171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:38.035171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:38.035171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:38.035171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:38.035171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:38.035171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:38.035171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:38.036094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:38.036094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:38.036094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:38.036094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:38.036094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:38.036094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:38.036094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:38.036094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:38.036094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:38.036094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:38.036094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:38.036094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:38.036094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:38.036094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:38.036094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:38.036094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:38.036829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:38.036829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:38.036829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:38.036829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:38.036829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:38.036829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:38.036829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:38.036829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:38.036829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:38.036829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:38.036829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:38.036829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:38.041007 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 64/131] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:38.042156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffd013-ef65-40b2-90b5-a414b7940538 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:38.047854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 68/132] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:12:37 2022] PUT /identity/v3/projects/3441a04d36074741b13ebfa058e656b8/users/907bde105eaf466980f8a555198f3c46/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:12:38.053546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362dd8a-00e3-473a-b1d5-ef2a49dcfe32 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:38.193685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffd013-ef65-40b2-90b5-a414b7940538 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:38.222610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:38.234025 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c728e68-3ef3-4b06-88fe-87482e6175b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:38.235083 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c728e68-3ef3-4b06-88fe-87482e6175b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:38.235561 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c728e68-3ef3-4b06-88fe-87482e6175b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:38.236144 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c728e68-3ef3-4b06-88fe-87482e6175b0 None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8/users/907bde105eaf466980f8a555198f3c46/roles/9f8a5b177f4a41fc8bc88ff4f1ce0839` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:38.238713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c728e68-3ef3-4b06-88fe-87482e6175b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=3441a04d36074741b13ebfa058e656b8, user_id=907bde105eaf466980f8a555198f3c46, role_id=9f8a5b177f4a41fc8bc88ff4f1ce0839)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:38.241232 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c728e68-3ef3-4b06-88fe-87482e6175b0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:38.257805 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362dd8a-00e3-473a-b1d5-ef2a49dcfe32 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:38.269013 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 65/133] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:12:38 2022] PUT /identity/v3/projects/3441a04d36074741b13ebfa058e656b8/users/907bde105eaf466980f8a555198f3c46/roles/9f8a5b177f4a41fc8bc88ff4f1ce0839 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:12:38.316679 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4859817-1cd9-4903-b757-7dc782c8ebd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:38.317317 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4859817-1cd9-4903-b757-7dc782c8ebd4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:38.317601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4859817-1cd9-4903-b757-7dc782c8ebd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:38.317876 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4859817-1cd9-4903-b757-7dc782c8ebd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:38.318655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4859817-1cd9-4903-b757-7dc782c8ebd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:38.318966 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4859817-1cd9-4903-b757-7dc782c8ebd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:38.320370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4859817-1cd9-4903-b757-7dc782c8ebd4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:38.337657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 69/134] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:12:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:38.343324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c728e68-3ef3-4b06-88fe-87482e6175b0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:38.460323 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c728e68-3ef3-4b06-88fe-87482e6175b0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:38.498092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be78b4f-0359-4762-8730-896985404097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:38.499118 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be78b4f-0359-4762-8730-896985404097 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:38.499591 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be78b4f-0359-4762-8730-896985404097 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:38.500050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be78b4f-0359-4762-8730-896985404097 None admin] PATH_INFO: `/v3/users/907bde105eaf466980f8a555198f3c46` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:38.501452 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be78b4f-0359-4762-8730-896985404097 None admin] RBAC: Authorizing `identity:get_user(user_id=907bde105eaf466980f8a555198f3c46)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:38.503185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be78b4f-0359-4762-8730-896985404097 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:38.505281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 66/135] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:38 2022] GET /identity/v3/users/907bde105eaf466980f8a555198f3c46 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:38.510531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4859817-1cd9-4903-b757-7dc782c8ebd4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:38.560562 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:38.665119 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4859817-1cd9-4903-b757-7dc782c8ebd4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:38.710606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ee8a1-1272-4d85-9068-345d8cfcda07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:38.711178 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ee8a1-1272-4d85-9068-345d8cfcda07 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:38.711463 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ee8a1-1272-4d85-9068-345d8cfcda07 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:38.711744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ee8a1-1272-4d85-9068-345d8cfcda07 None admin] PATH_INFO: `/v3/projects/27caf57dbf92480283ca4dd69f9d0a43` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:38.712913 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ee8a1-1272-4d85-9068-345d8cfcda07 None admin] RBAC: Authorizing `identity:get_project(project_id=27caf57dbf92480283ca4dd69f9d0a43)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:38.714396 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ee8a1-1272-4d85-9068-345d8cfcda07 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:38.716051 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 70/136] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:38 2022] GET /identity/v3/projects/27caf57dbf92480283ca4dd69f9d0a43 => generated 312 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:38.721878 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be78b4f-0359-4762-8730-896985404097 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:38.798530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:38.856743 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be78b4f-0359-4762-8730-896985404097 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:38.903840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253b8fb-a525-4fc8-a1a9-2c9b62b4cff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:38.904880 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253b8fb-a525-4fc8-a1a9-2c9b62b4cff8 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:38.905358 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253b8fb-a525-4fc8-a1a9-2c9b62b4cff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:38.905820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253b8fb-a525-4fc8-a1a9-2c9b62b4cff8 None admin] PATH_INFO: `/v3/projects/27caf57dbf92480283ca4dd69f9d0a43/users/907bde105eaf466980f8a555198f3c46/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:38.908066 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253b8fb-a525-4fc8-a1a9-2c9b62b4cff8 None admin] RBAC: Authorizing `identity:create_grant(project_id=27caf57dbf92480283ca4dd69f9d0a43, user_id=907bde105eaf466980f8a555198f3c46, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:38.909899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253b8fb-a525-4fc8-a1a9-2c9b62b4cff8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:38.936254 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 67/137] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:12:38 2022] PUT /identity/v3/projects/27caf57dbf92480283ca4dd69f9d0a43/users/907bde105eaf466980f8a555198f3c46/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:12:39.359064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee940083-23c7-4a40-8645-3c09c1a2db3f None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:39.359443 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee940083-23c7-4a40-8645-3c09c1a2db3f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:39.359620 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee940083-23c7-4a40-8645-3c09c1a2db3f None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:39.361112 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 71/138] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:39.367159 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf62e50-2d50-4ed0-b32b-1bc823d9f6c1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:39.367719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf62e50-2d50-4ed0-b32b-1bc823d9f6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:39.368221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf62e50-2d50-4ed0-b32b-1bc823d9f6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:39.387171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ebf62e50-2d50-4ed0-b32b-1bc823d9f6c1 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:39.388687 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ebf62e50-2d50-4ed0-b32b-1bc823d9f6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:39.462344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 68/139] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:39.487119 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee940083-23c7-4a40-8645-3c09c1a2db3f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:39.599202 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:39.603536 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee940083-23c7-4a40-8645-3c09c1a2db3f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:39.639731 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3201a6-ed27-4f8a-af15-36aebfb42433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:39.640282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3201a6-ed27-4f8a-af15-36aebfb42433 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:39.640567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3201a6-ed27-4f8a-af15-36aebfb42433 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:39.640844 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3201a6-ed27-4f8a-af15-36aebfb42433 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:39.641962 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3201a6-ed27-4f8a-af15-36aebfb42433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:39.643500 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3201a6-ed27-4f8a-af15-36aebfb42433 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:39.645341 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 72/140] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:12:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:39.650765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf62e50-2d50-4ed0-b32b-1bc823d9f6c1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:39.704444 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:39.769479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf62e50-2d50-4ed0-b32b-1bc823d9f6c1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:39.808427 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fb11c-e10d-48c6-a564-7ef579913620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:39.809708 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fb11c-e10d-48c6-a564-7ef579913620 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:39.810179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fb11c-e10d-48c6-a564-7ef579913620 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:39.810665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fb11c-e10d-48c6-a564-7ef579913620 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:39.811507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232fb11c-e10d-48c6-a564-7ef579913620 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:39.813205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232fb11c-e10d-48c6-a564-7ef579913620 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:39.843156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 69/141] 10.222.0.39 () {60 vars in 1132 bytes} [Mon Aug 8 02:12:39 2022] POST /identity/v3/users => generated 283 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:12:40.299969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bd991-94eb-4252-884d-722626c7823f None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:40.300337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bd991-94eb-4252-884d-722626c7823f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:40.300536 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bd991-94eb-4252-884d-722626c7823f None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:40.302155 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 73/142] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:40.307838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe4bd0-ba15-4af6-8fce-b12081863f3f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:40.308440 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe4bd0-ba15-4af6-8fce-b12081863f3f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:40.308907 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe4bd0-ba15-4af6-8fce-b12081863f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:40.334958 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5fbe4bd0-ba15-4af6-8fce-b12081863f3f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:40.336465 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5fbe4bd0-ba15-4af6-8fce-b12081863f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:40.416520 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 70/143] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:40.452271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bd991-94eb-4252-884d-722626c7823f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:40.586714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bd991-94eb-4252-884d-722626c7823f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:40.632824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a603bc-8988-4d68-9671-61edef440799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:40.634266 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a603bc-8988-4d68-9671-61edef440799 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:40.634737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a603bc-8988-4d68-9671-61edef440799 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:40.635260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a603bc-8988-4d68-9671-61edef440799 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:40.648698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a603bc-8988-4d68-9671-61edef440799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:40.653671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-70a603bc-8988-4d68-9671-61edef440799 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:12:40.653671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:40.653671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:40.653671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:40.653671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:40.653671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:40.653671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:40.653671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:40.653671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:40.653671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:40.653671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:40.653671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:40.653671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:40.653671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:40.653671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:40.653671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:40.654520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:40.654520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:40.654520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:40.654520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:40.654520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:40.654520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:40.654520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:40.654520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:40.654520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:40.654520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:40.654520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:40.654520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:40.654520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:40.654520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:40.654520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:40.654520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:40.655247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:40.655247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:40.655247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:40.655247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:40.655247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:40.655247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:40.655247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:40.655247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:40.655247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:40.655247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:40.655247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:12:40.655247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:12:40.657484 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 74/144] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:40.663638 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe4bd0-ba15-4af6-8fce-b12081863f3f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:40.698495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:40.734129 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31087cf-9631-4a0c-9e7f-2ab8499b163a None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:40.734459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31087cf-9631-4a0c-9e7f-2ab8499b163a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:40.734725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31087cf-9631-4a0c-9e7f-2ab8499b163a None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:40.736349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 75/145] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:40.742144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee8fe7-526b-4d52-b6fe-437913bf34b5 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:40.742483 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee8fe7-526b-4d52-b6fe-437913bf34b5 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:40.742796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee8fe7-526b-4d52-b6fe-437913bf34b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:40.773960 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6eee8fe7-526b-4d52-b6fe-437913bf34b5 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:40.775212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6eee8fe7-526b-4d52-b6fe-437913bf34b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:40.820762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe4bd0-ba15-4af6-8fce-b12081863f3f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:40.830634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:40.857528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8aafb-a994-4f7d-8b62-ad04df3fd53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:40.858059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8aafb-a994-4f7d-8b62-ad04df3fd53f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:40.858370 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8aafb-a994-4f7d-8b62-ad04df3fd53f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:40.858644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8aafb-a994-4f7d-8b62-ad04df3fd53f None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:40.859316 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8aafb-a994-4f7d-8b62-ad04df3fd53f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:40.859619 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8aafb-a994-4f7d-8b62-ad04df3fd53f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:40.861068 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8aafb-a994-4f7d-8b62-ad04df3fd53f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:40.871375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 71/146] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:12:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:40.877216 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8aafb-a994-4f7d-8b62-ad04df3fd53f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:40.944850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 76/147] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:40.971573 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee8fe7-526b-4d52-b6fe-437913bf34b5 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:40.977548 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:40.986919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8aafb-a994-4f7d-8b62-ad04df3fd53f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:41.022120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc21fb3-0655-46c0-9e89-cbc8ef703662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:41.022782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc21fb3-0655-46c0-9e89-cbc8ef703662 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:41.023059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc21fb3-0655-46c0-9e89-cbc8ef703662 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:41.023382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc21fb3-0655-46c0-9e89-cbc8ef703662 None admin] PATH_INFO: `/v3/users/8ef3ff73ef554c22aa1dec9bab091558` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:41.024567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc21fb3-0655-46c0-9e89-cbc8ef703662 None admin] RBAC: Authorizing `identity:get_user(user_id=8ef3ff73ef554c22aa1dec9bab091558)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:41.026012 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc21fb3-0655-46c0-9e89-cbc8ef703662 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:41.027619 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 72/148] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:40 2022] GET /identity/v3/users/8ef3ff73ef554c22aa1dec9bab091558 => generated 289 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:41.033145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc21fb3-0655-46c0-9e89-cbc8ef703662 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:41.144607 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc21fb3-0655-46c0-9e89-cbc8ef703662 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:41.172154 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee8fe7-526b-4d52-b6fe-437913bf34b5 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:41.181462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86d47b-f506-4f3d-904c-ec0328619628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:41.182095 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86d47b-f506-4f3d-904c-ec0328619628 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:41.182583 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86d47b-f506-4f3d-904c-ec0328619628 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:41.182853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86d47b-f506-4f3d-904c-ec0328619628 None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:41.183976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86d47b-f506-4f3d-904c-ec0328619628 None admin] RBAC: Authorizing `identity:get_project(project_id=3441a04d36074741b13ebfa058e656b8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:41.185752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86d47b-f506-4f3d-904c-ec0328619628 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:41.187412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 73/149] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:41 2022] GET /identity/v3/projects/3441a04d36074741b13ebfa058e656b8 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:41.193253 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86d47b-f506-4f3d-904c-ec0328619628 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:41.230606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:41.238989 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef898cb-b250-4d82-8bd1-28f6b5e13a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:41.239641 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef898cb-b250-4d82-8bd1-28f6b5e13a34 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:41.239922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef898cb-b250-4d82-8bd1-28f6b5e13a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:41.240194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef898cb-b250-4d82-8bd1-28f6b5e13a34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:41.259862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef898cb-b250-4d82-8bd1-28f6b5e13a34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:41.265524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-5ef898cb-b250-4d82-8bd1-28f6b5e13a34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:12:41.265524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:41.265524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:41.265524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:41.265524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:41.265524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:41.265524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:41.265524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:41.265524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:41.265524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:41.265524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:41.265524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:41.265524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:41.265524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:41.265524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:41.265524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:41.266425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:41.266425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:41.266425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:41.266425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:41.266425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:41.266425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:41.266425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:41.266425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:41.266425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:41.266425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:41.266425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:41.266425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:41.266425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:41.266425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:41.266425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:41.266425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:41.267177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:41.267177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:41.267177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:41.267177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:41.267177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:41.267177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:41.267177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:41.267177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:41.267177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:41.267177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:41.267177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:12:41.267177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:12:41.268467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 77/150] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:12:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:12:41.275010 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef898cb-b250-4d82-8bd1-28f6b5e13a34 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:41.316643 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:41.347578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86d47b-f506-4f3d-904c-ec0328619628 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:41.394975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14b1de-6643-4305-9043-6f4388bee9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:41.395979 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14b1de-6643-4305-9043-6f4388bee9e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:41.396462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14b1de-6643-4305-9043-6f4388bee9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:41.397478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14b1de-6643-4305-9043-6f4388bee9e2 None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8/users/8ef3ff73ef554c22aa1dec9bab091558/roles/3b1d23e3e45d44a3a8069b8e823410b5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:41.403191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14b1de-6643-4305-9043-6f4388bee9e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3441a04d36074741b13ebfa058e656b8, user_id=8ef3ff73ef554c22aa1dec9bab091558, role_id=3b1d23e3e45d44a3a8069b8e823410b5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:41.405102 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14b1de-6643-4305-9043-6f4388bee9e2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:41.427539 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef898cb-b250-4d82-8bd1-28f6b5e13a34 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:41.437556 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 74/151] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:12:41 2022] PUT /identity/v3/projects/3441a04d36074741b13ebfa058e656b8/users/8ef3ff73ef554c22aa1dec9bab091558/roles/3b1d23e3e45d44a3a8069b8e823410b5 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:12:41.472986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818f7b7-aedc-40ff-8547-af6b34b59ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:41.473611 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818f7b7-aedc-40ff-8547-af6b34b59ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:41.474011 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818f7b7-aedc-40ff-8547-af6b34b59ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:41.474304 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818f7b7-aedc-40ff-8547-af6b34b59ba0 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:41.475567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818f7b7-aedc-40ff-8547-af6b34b59ba0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:41.475873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818f7b7-aedc-40ff-8547-af6b34b59ba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:41.477316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818f7b7-aedc-40ff-8547-af6b34b59ba0 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:41.490689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 78/152] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:41.493325 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14b1de-6643-4305-9043-6f4388bee9e2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:41.566277 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df4a14-c2b2-4443-b99a-0b9214c72124 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:41.566598 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df4a14-c2b2-4443-b99a-0b9214c72124 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:41.566829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df4a14-c2b2-4443-b99a-0b9214c72124 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:41.568448 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 79/153] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:41.578364 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32f05b-6440-4b2e-8580-5a9a14fb58b4 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:41.578660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32f05b-6440-4b2e-8580-5a9a14fb58b4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:41.578947 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32f05b-6440-4b2e-8580-5a9a14fb58b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:41.611394 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-dc32f05b-6440-4b2e-8580-5a9a14fb58b4 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:41.612357 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:41.612602 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dc32f05b-6440-4b2e-8580-5a9a14fb58b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:41.629635 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14b1de-6643-4305-9043-6f4388bee9e2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:41.681506 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53dcd0-fd38-46a1-b2fb-02ace0a9a6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:41.682132 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53dcd0-fd38-46a1-b2fb-02ace0a9a6aa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:41.682461 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53dcd0-fd38-46a1-b2fb-02ace0a9a6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:41.682644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53dcd0-fd38-46a1-b2fb-02ace0a9a6aa None admin] PATH_INFO: `/v3/users/2d716e3acbb54b2fa8e10b30ba8eb5fb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:41.687057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53dcd0-fd38-46a1-b2fb-02ace0a9a6aa None admin] RBAC: Authorizing `identity:get_user(user_id=2d716e3acbb54b2fa8e10b30ba8eb5fb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:41.688654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53dcd0-fd38-46a1-b2fb-02ace0a9a6aa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:41.690453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 75/154] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:41 2022] GET /identity/v3/users/2d716e3acbb54b2fa8e10b30ba8eb5fb => generated 247 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:41.703598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53dcd0-fd38-46a1-b2fb-02ace0a9a6aa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:41.716896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 80/155] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:41.756250 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32f05b-6440-4b2e-8580-5a9a14fb58b4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:41.773022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:41.855441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53dcd0-fd38-46a1-b2fb-02ace0a9a6aa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:41.900458 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a29bc-9d24-4cc9-9d44-1d29cc092ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:41.901082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a29bc-9d24-4cc9-9d44-1d29cc092ecd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:41.901362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a29bc-9d24-4cc9-9d44-1d29cc092ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:41.901636 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a29bc-9d24-4cc9-9d44-1d29cc092ecd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:41.904895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32f05b-6440-4b2e-8580-5a9a14fb58b4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:41.938933 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:41.962974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767bb0a-5cd6-4905-8fcf-f92b732c04a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:41.963629 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767bb0a-5cd6-4905-8fcf-f92b732c04a0 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:41.963907 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767bb0a-5cd6-4905-8fcf-f92b732c04a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:41.964175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767bb0a-5cd6-4905-8fcf-f92b732c04a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:41.979653 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767bb0a-5cd6-4905-8fcf-f92b732c04a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:41.984460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-9767bb0a-5cd6-4905-8fcf-f92b732c04a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:41.984460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:41.984460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:41.984460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:41.984460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:41.984460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:41.984460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:41.984460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:41.984460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:41.984460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:41.984460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:41.984460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:41.984460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:41.984460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:41.984460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:41.984460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:41.985466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:41.985466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:41.985466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:41.985466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:41.985466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:41.985466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:41.985466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:41.985466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:41.985466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:41.985466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:41.985466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:41.985466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:41.985466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:41.985466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:41.985466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:41.985466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:41.986330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:41.986330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:41.986330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:41.986330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:41.986330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:41.986330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:41.986330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:41.986330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:41.986330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:41.986330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:41.986330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:41.986330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:12:41.986330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 81/156] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:12:41.992359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a29bc-9d24-4cc9-9d44-1d29cc092ecd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:41.993996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a29bc-9d24-4cc9-9d44-1d29cc092ecd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:41.995220 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767bb0a-5cd6-4905-8fcf-f92b732c04a0 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:42.094817 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:42.128385 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 76/157] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:12:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:12:42.140438 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767bb0a-5cd6-4905-8fcf-f92b732c04a0 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:42.145477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a29bc-9d24-4cc9-9d44-1d29cc092ecd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:42.184133 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b25aa-fd98-422f-935e-d6097b577839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:42.184994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b25aa-fd98-422f-935e-d6097b577839 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:42.185267 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b25aa-fd98-422f-935e-d6097b577839 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:42.185532 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b25aa-fd98-422f-935e-d6097b577839 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:42.186201 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b25aa-fd98-422f-935e-d6097b577839 None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:42.186522 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b25aa-fd98-422f-935e-d6097b577839 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:42.187909 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b25aa-fd98-422f-935e-d6097b577839 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:42.197841 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 82/158] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:12:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:42.203754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b25aa-fd98-422f-935e-d6097b577839 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:42.207857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:42.255341 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a29bc-9d24-4cc9-9d44-1d29cc092ecd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:42.299348 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63dc9b6-61b1-4f6b-a4b8-78c9ccd4f7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:42.300190 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63dc9b6-61b1-4f6b-a4b8-78c9ccd4f7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:42.300595 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63dc9b6-61b1-4f6b-a4b8-78c9ccd4f7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:42.300878 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63dc9b6-61b1-4f6b-a4b8-78c9ccd4f7d4 None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:42.302039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63dc9b6-61b1-4f6b-a4b8-78c9ccd4f7d4 None admin] RBAC: Authorizing `identity:get_project(project_id=3441a04d36074741b13ebfa058e656b8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:42.304356 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63dc9b6-61b1-4f6b-a4b8-78c9ccd4f7d4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:42.306345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 77/159] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:42 2022] GET /identity/v3/projects/3441a04d36074741b13ebfa058e656b8 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:42.311923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63dc9b6-61b1-4f6b-a4b8-78c9ccd4f7d4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:42.356109 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b25aa-fd98-422f-935e-d6097b577839 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:42.388103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:42.403998 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1db69-fbc7-4c86-a92a-e870d4022802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:42.404635 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1db69-fbc7-4c86-a92a-e870d4022802 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:42.404914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1db69-fbc7-4c86-a92a-e870d4022802 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:42.405184 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1db69-fbc7-4c86-a92a-e870d4022802 None admin] PATH_INFO: `/v3/users/907bde105eaf466980f8a555198f3c46` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:42.407607 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1db69-fbc7-4c86-a92a-e870d4022802 None admin] RBAC: Authorizing `identity:get_user(user_id=907bde105eaf466980f8a555198f3c46)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:42.409075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1db69-fbc7-4c86-a92a-e870d4022802 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:42.411085 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 83/160] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:42 2022] GET /identity/v3/users/907bde105eaf466980f8a555198f3c46 => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:42.416898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1db69-fbc7-4c86-a92a-e870d4022802 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:42.436998 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63dc9b6-61b1-4f6b-a4b8-78c9ccd4f7d4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:42.467313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:42.479307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b61ea8-af55-4141-9765-642c0a00a4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:42.480076 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b61ea8-af55-4141-9765-642c0a00a4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:42.480448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b61ea8-af55-4141-9765-642c0a00a4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:42.480829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b61ea8-af55-4141-9765-642c0a00a4b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:42.481639 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b61ea8-af55-4141-9765-642c0a00a4b8 None admin] RBAC: Adding query filter params (role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.project.id=3441a04d36074741b13ebfa058e656b8, user.id=2d716e3acbb54b2fa8e10b30ba8eb5fb) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:42.482120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b61ea8-af55-4141-9765-642c0a00a4b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:42.483742 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b61ea8-af55-4141-9765-642c0a00a4b8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:42.491266 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 78/161] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:12:42 2022] GET /identity/v3/role_assignments?user.id=2d716e3acbb54b2fa8e10b30ba8eb5fb&scope.project.id=3441a04d36074741b13ebfa058e656b8&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:42.577812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1db69-fbc7-4c86-a92a-e870d4022802 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:42.589710 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738416d8-398b-41ba-bc11-00584de7b0fe None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:42.590027 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738416d8-398b-41ba-bc11-00584de7b0fe None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:42.590276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738416d8-398b-41ba-bc11-00584de7b0fe None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:42.591783 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 79/162] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:42.600237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a24e23-b41b-4c3d-97f3-f2768c23354c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:42.600549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a24e23-b41b-4c3d-97f3-f2768c23354c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:42.600778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a24e23-b41b-4c3d-97f3-f2768c23354c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:42.629534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc91a8d-5d60-4e8c-b8a9-dd04d3957c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:42.630149 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc91a8d-5d60-4e8c-b8a9-dd04d3957c5e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:42.630448 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc91a8d-5d60-4e8c-b8a9-dd04d3957c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:42.630719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc91a8d-5d60-4e8c-b8a9-dd04d3957c5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:42.632504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c9a24e23-b41b-4c3d-97f3-f2768c23354c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:42.633729 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c9a24e23-b41b-4c3d-97f3-f2768c23354c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:42.709344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 80/163] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:42.710542 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:42.711590 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041f4f0-65c3-40b5-8bb3-a2ae4608b80e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:42.711885 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041f4f0-65c3-40b5-8bb3-a2ae4608b80e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:42.712171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041f4f0-65c3-40b5-8bb3-a2ae4608b80e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:42.713592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 81/164] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:42.716052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc91a8d-5d60-4e8c-b8a9-dd04d3957c5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:42.717635 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc91a8d-5d60-4e8c-b8a9-dd04d3957c5e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:42.719224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9d6ad-c470-46a0-b1d6-72fd79e9549d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:42.719603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9d6ad-c470-46a0-b1d6-72fd79e9549d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:42.719982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9d6ad-c470-46a0-b1d6-72fd79e9549d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:42.755464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-92f9d6ad-c470-46a0-b1d6-72fd79e9549d None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:42.756729 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-92f9d6ad-c470-46a0-b1d6-72fd79e9549d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:42.827351 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 84/165] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:12:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:12:42.831231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc91a8d-5d60-4e8c-b8a9-dd04d3957c5e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:42.867209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 82/166] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:42.868075 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9d6ad-c470-46a0-b1d6-72fd79e9549d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:42.958662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:42.965251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc91a8d-5d60-4e8c-b8a9-dd04d3957c5e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:43.015333 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ed25e-69ac-41a9-b425-21166f1379cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:43.015965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ed25e-69ac-41a9-b425-21166f1379cd None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:43.016247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ed25e-69ac-41a9-b425-21166f1379cd None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:43.016523 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ed25e-69ac-41a9-b425-21166f1379cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:43.036798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ed25e-69ac-41a9-b425-21166f1379cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:43.044307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-ea3ed25e-69ac-41a9-b425-21166f1379cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:43.044307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:43.044307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:43.044307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:43.044307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:43.044307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:43.044307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:43.044307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:43.044307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:43.044307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:43.044307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:43.044307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:43.044307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:43.044307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:43.044307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:43.044307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:43.045252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:43.045252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:43.045252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:43.045252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:43.045252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:43.045252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:43.045252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:43.045252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:43.045252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:43.045252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:43.045252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:43.045252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:43.045252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:43.045252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:43.045252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:43.045252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:43.046318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:43.046318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:43.046318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:43.046318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:43.046318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:43.046318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:43.046318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:43.046318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:43.046318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:43.046318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:43.046318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:43.046318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:12:43.046318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 85/167] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:43.052033 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ed25e-69ac-41a9-b425-21166f1379cd None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:43.052555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:43.188361 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9d6ad-c470-46a0-b1d6-72fd79e9549d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:43.204377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ed25e-69ac-41a9-b425-21166f1379cd None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:43.249283 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cc3d4-4e87-442f-b071-f6755355107e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:43.249907 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cc3d4-4e87-442f-b071-f6755355107e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:43.250185 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cc3d4-4e87-442f-b071-f6755355107e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:43.250493 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cc3d4-4e87-442f-b071-f6755355107e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:43.260630 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871cc3d4-4e87-442f-b071-f6755355107e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:43.265558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-871cc3d4-4e87-442f-b071-f6755355107e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:12:43.265558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:43.265558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:43.265558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:43.265558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:43.265558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:43.265558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:43.265558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:43.265558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:43.265558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:43.265558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:43.265558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:43.265558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:43.265558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:43.265558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:43.265558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:43.266427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:43.266427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:43.266427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:43.266427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:43.266427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:43.266427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:43.266427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:43.266427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:43.266427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:43.266427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:43.266427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:43.266427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:43.266427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:43.266427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:43.266427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:43.266427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:43.267193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:43.267193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:43.267193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:43.267193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:43.267193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:43.267193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:43.267193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:43.267193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:43.267193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:43.267193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:43.267193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:12:43.267193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:12:43.267193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 86/168] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:12:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:43.267905 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cc3d4-4e87-442f-b071-f6755355107e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:43.278932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35da991-1d2f-4a65-b04e-3285125c33d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:43.279883 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35da991-1d2f-4a65-b04e-3285125c33d4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:43.280356 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35da991-1d2f-4a65-b04e-3285125c33d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:43.280818 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35da991-1d2f-4a65-b04e-3285125c33d4 None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:43.286944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35da991-1d2f-4a65-b04e-3285125c33d4 None admin] RBAC: Authorizing `identity:get_project(project_id=3441a04d36074741b13ebfa058e656b8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:43.289175 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35da991-1d2f-4a65-b04e-3285125c33d4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:43.291324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 83/169] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:42 2022] GET /identity/v3/projects/3441a04d36074741b13ebfa058e656b8 => generated 298 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:43.292293 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35da991-1d2f-4a65-b04e-3285125c33d4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:43.310734 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:43.369165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cc3d4-4e87-442f-b071-f6755355107e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:43.404483 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb2d8c-ccf5-4fe2-8f4c-a30f807609e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:43.405085 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb2d8c-ccf5-4fe2-8f4c-a30f807609e4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:43.405411 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb2d8c-ccf5-4fe2-8f4c-a30f807609e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:43.405680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb2d8c-ccf5-4fe2-8f4c-a30f807609e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:43.406401 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb2d8c-ccf5-4fe2-8f4c-a30f807609e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:43.406703 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb2d8c-ccf5-4fe2-8f4c-a30f807609e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:43.410270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb2d8c-ccf5-4fe2-8f4c-a30f807609e4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:43.416736 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 87/170] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:12:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:43.421796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb2d8c-ccf5-4fe2-8f4c-a30f807609e4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:43.510593 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35da991-1d2f-4a65-b04e-3285125c33d4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:43.510820 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:43.529190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb2d8c-ccf5-4fe2-8f4c-a30f807609e4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:43.540601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:43.575889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1e994-6551-4ffa-a7f8-80519c6667bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:43.577276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1e994-6551-4ffa-a7f8-80519c6667bf None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:43.578029 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1e994-6551-4ffa-a7f8-80519c6667bf None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:43.579026 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1e994-6551-4ffa-a7f8-80519c6667bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:43.579817 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08280550-9e85-4c04-807c-e1caa0b30210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:43.580742 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1e994-6551-4ffa-a7f8-80519c6667bf None admin] RBAC: Adding query filter params (role.id=42c846f6944b40c8af5eeb8360cb7c7a, scope.project.id=3441a04d36074741b13ebfa058e656b8, user.id=907bde105eaf466980f8a555198f3c46) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:43.581525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08280550-9e85-4c04-807c-e1caa0b30210 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:43.582018 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1e994-6551-4ffa-a7f8-80519c6667bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:43.586693 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08280550-9e85-4c04-807c-e1caa0b30210 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:43.588368 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1e994-6551-4ffa-a7f8-80519c6667bf None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:43.588868 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08280550-9e85-4c04-807c-e1caa0b30210 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:43.589759 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08280550-9e85-4c04-807c-e1caa0b30210 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:43.590259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08280550-9e85-4c04-807c-e1caa0b30210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:43.594103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08280550-9e85-4c04-807c-e1caa0b30210 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:43.607103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 88/171] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:12:43 2022] GET /identity/v3/role_assignments?user.id=907bde105eaf466980f8a555198f3c46&scope.project.id=3441a04d36074741b13ebfa058e656b8&role.id=42c846f6944b40c8af5eeb8360cb7c7a => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:43.610218 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1e994-6551-4ffa-a7f8-80519c6667bf None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:43.620417 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 84/172] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:12:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:43.631925 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08280550-9e85-4c04-807c-e1caa0b30210 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:43.805853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08280550-9e85-4c04-807c-e1caa0b30210 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:43.837925 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1e994-6551-4ffa-a7f8-80519c6667bf None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:43.867056 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6b518-a84b-41ee-8ea6-b98b4746ee94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:43.867722 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6b518-a84b-41ee-8ea6-b98b4746ee94 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:43.868006 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6b518-a84b-41ee-8ea6-b98b4746ee94 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:43.868281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6b518-a84b-41ee-8ea6-b98b4746ee94 None admin] PATH_INFO: `/v3/users/907bde105eaf466980f8a555198f3c46` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:43.869526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b6b518-a84b-41ee-8ea6-b98b4746ee94 None admin] RBAC: Authorizing `identity:get_user(user_id=907bde105eaf466980f8a555198f3c46)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:43.871049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b6b518-a84b-41ee-8ea6-b98b4746ee94 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:43.872684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 85/173] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:43 2022] GET /identity/v3/users/907bde105eaf466980f8a555198f3c46 => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:43.881456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6b518-a84b-41ee-8ea6-b98b4746ee94 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:43.908956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2b259-6dc3-4146-9302-b4f3bf9f57db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:43.909600 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2b259-6dc3-4146-9302-b4f3bf9f57db None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:43.909880 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2b259-6dc3-4146-9302-b4f3bf9f57db None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:43.910155 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2b259-6dc3-4146-9302-b4f3bf9f57db None admin] PATH_INFO: `/v3/users/907bde105eaf466980f8a555198f3c46` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:43.912481 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2b259-6dc3-4146-9302-b4f3bf9f57db None admin] RBAC: Authorizing `identity:get_user(user_id=907bde105eaf466980f8a555198f3c46)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:43.913934 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2b259-6dc3-4146-9302-b4f3bf9f57db None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:43.915982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 89/174] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:43 2022] GET /identity/v3/users/907bde105eaf466980f8a555198f3c46 => generated 275 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:43.921478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2b259-6dc3-4146-9302-b4f3bf9f57db None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:43.947662 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:43.949463 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:44.003812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6b518-a84b-41ee-8ea6-b98b4746ee94 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:44.040094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c599859-f3bb-4157-a04c-d6613daad4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:44.040797 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c599859-f3bb-4157-a04c-d6613daad4bb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:44.041148 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c599859-f3bb-4157-a04c-d6613daad4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:44.041487 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c599859-f3bb-4157-a04c-d6613daad4bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:44.102271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2b259-6dc3-4146-9302-b4f3bf9f57db None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:44.111606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c599859-f3bb-4157-a04c-d6613daad4bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:44.113184 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c599859-f3bb-4157-a04c-d6613daad4bb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:44.159481 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:44.165047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b863952-f627-42ad-9958-48a32e478924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:44.165662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b863952-f627-42ad-9958-48a32e478924 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:44.165944 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b863952-f627-42ad-9958-48a32e478924 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:44.166298 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b863952-f627-42ad-9958-48a32e478924 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:44.215925 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 86/175] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:12:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:12:44.224043 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c599859-f3bb-4157-a04c-d6613daad4bb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:44.285533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b863952-f627-42ad-9958-48a32e478924 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:44.288753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b863952-f627-42ad-9958-48a32e478924 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:44.314312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:44.416761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c599859-f3bb-4157-a04c-d6613daad4bb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:44.480222 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 90/176] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:12:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 02:12:44.482161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f051210-c8c8-4df0-a8b8-4ec167f4ddde None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:44.482474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f051210-c8c8-4df0-a8b8-4ec167f4ddde None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:44.482755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f051210-c8c8-4df0-a8b8-4ec167f4ddde None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:44.484166 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 91/177] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:44.487142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f051210-c8c8-4df0-a8b8-4ec167f4ddde None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:44.489897 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750e9b5-4cf0-4c4f-9fa1-8063c0fadd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:44.490632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750e9b5-4cf0-4c4f-9fa1-8063c0fadd86 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:44.490911 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750e9b5-4cf0-4c4f-9fa1-8063c0fadd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:44.491181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750e9b5-4cf0-4c4f-9fa1-8063c0fadd86 None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:44.494003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750e9b5-4cf0-4c4f-9fa1-8063c0fadd86 None admin] RBAC: Authorizing `identity:get_project(project_id=3441a04d36074741b13ebfa058e656b8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:44.495570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750e9b5-4cf0-4c4f-9fa1-8063c0fadd86 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:44.497225 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 87/178] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:44 2022] GET /identity/v3/projects/3441a04d36074741b13ebfa058e656b8 => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:44.498377 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f61b0-0eea-4679-92d9-8434e8a87246 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:44.498948 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f61b0-0eea-4679-92d9-8434e8a87246 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:44.499222 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f61b0-0eea-4679-92d9-8434e8a87246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:44.536180 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-286f61b0-0eea-4679-92d9-8434e8a87246 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:44.537939 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-286f61b0-0eea-4679-92d9-8434e8a87246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:44.562322 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:44.655821 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 88/179] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:44.656692 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f61b0-0eea-4679-92d9-8434e8a87246 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:44.689508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f051210-c8c8-4df0-a8b8-4ec167f4ddde None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:44.704624 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:44.727856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d1b88-5a2a-4534-a542-f57d787997c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:44.728548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d1b88-5a2a-4534-a542-f57d787997c8 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:44.728831 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d1b88-5a2a-4534-a542-f57d787997c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:44.729122 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d1b88-5a2a-4534-a542-f57d787997c8 None admin] PATH_INFO: `/v3/projects/27caf57dbf92480283ca4dd69f9d0a43` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:44.730468 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d1b88-5a2a-4534-a542-f57d787997c8 None admin] RBAC: Authorizing `identity:get_project(project_id=27caf57dbf92480283ca4dd69f9d0a43)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:44.732011 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d1b88-5a2a-4534-a542-f57d787997c8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:44.733924 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 92/180] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:44 2022] GET /identity/v3/projects/27caf57dbf92480283ca4dd69f9d0a43 => generated 312 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:44.734729 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d1b88-5a2a-4534-a542-f57d787997c8 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:44.830440 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f61b0-0eea-4679-92d9-8434e8a87246 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:44.841768 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d1b88-5a2a-4534-a542-f57d787997c8 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:44.879319 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e24c97-3735-485c-913e-2bd9103dba72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:44.880525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72dbcf-4c64-4e38-8f00-d15f421e67ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:44.881381 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e24c97-3735-485c-913e-2bd9103dba72 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:44.881668 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e24c97-3735-485c-913e-2bd9103dba72 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:44.881937 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e24c97-3735-485c-913e-2bd9103dba72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:44.882705 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e24c97-3735-485c-913e-2bd9103dba72 None admin] RBAC: Adding query filter params (role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839, scope.project.id=3441a04d36074741b13ebfa058e656b8, user.id=907bde105eaf466980f8a555198f3c46) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:44.883015 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e24c97-3735-485c-913e-2bd9103dba72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:44.883550 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72dbcf-4c64-4e38-8f00-d15f421e67ef None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:44.883823 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72dbcf-4c64-4e38-8f00-d15f421e67ef None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:44.884084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72dbcf-4c64-4e38-8f00-d15f421e67ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:44.885898 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e24c97-3735-485c-913e-2bd9103dba72 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:44.898052 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 89/181] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:12:44 2022] GET /identity/v3/role_assignments?user.id=907bde105eaf466980f8a555198f3c46&scope.project.id=3441a04d36074741b13ebfa058e656b8&role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:44.898745 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e24c97-3735-485c-913e-2bd9103dba72 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:44.905141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72dbcf-4c64-4e38-8f00-d15f421e67ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:44.914146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-fb72dbcf-4c64-4e38-8f00-d15f421e67ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:12:44.914146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:44.914146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:44.914146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:44.914146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:44.914146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:44.914146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:44.914146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:44.914146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:44.914146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:44.914146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:44.914146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:44.914146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:44.914146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:44.914146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:44.914146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:44.915123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:44.915123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:44.915123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:44.915123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:44.915123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:44.915123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:44.915123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:44.915123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:44.915123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:44.915123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:44.915123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:44.915123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:44.915123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:44.915123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:44.915123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:44.915123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:44.915898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:44.915898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:44.915898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:44.915898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:44.915898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:44.915898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:44.915898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:44.915898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:44.915898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:44.915898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:44.915898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:12:44.915898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:12:44.918445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 93/182] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:12:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:44.922903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:44.927413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72dbcf-4c64-4e38-8f00-d15f421e67ef None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:45.001646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:45.097672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72dbcf-4c64-4e38-8f00-d15f421e67ef None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:45.098870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e24c97-3735-485c-913e-2bd9103dba72 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:45.140251 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f500e7-5b99-4940-9fc3-e9bdc0b7b347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:45.141096 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f500e7-5b99-4940-9fc3-e9bdc0b7b347 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:45.141531 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f500e7-5b99-4940-9fc3-e9bdc0b7b347 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:45.141951 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f500e7-5b99-4940-9fc3-e9bdc0b7b347 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:45.142725 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f500e7-5b99-4940-9fc3-e9bdc0b7b347 None admin] RBAC: Adding query filter params (role.id=42c846f6944b40c8af5eeb8360cb7c7a, scope.project.id=27caf57dbf92480283ca4dd69f9d0a43, user.id=907bde105eaf466980f8a555198f3c46) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:45.143034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f500e7-5b99-4940-9fc3-e9bdc0b7b347 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:45.144191 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc7774-90ca-4716-9902-46c7948186f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:45.144475 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f500e7-5b99-4940-9fc3-e9bdc0b7b347 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:45.144790 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc7774-90ca-4716-9902-46c7948186f4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:45.145145 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc7774-90ca-4716-9902-46c7948186f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:45.145461 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc7774-90ca-4716-9902-46c7948186f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:45.146291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc7774-90ca-4716-9902-46c7948186f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:45.147259 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc7774-90ca-4716-9902-46c7948186f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:45.150278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc7774-90ca-4716-9902-46c7948186f4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:45.153011 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 90/183] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:12:44 2022] GET /identity/v3/role_assignments?user.id=907bde105eaf466980f8a555198f3c46&scope.project.id=27caf57dbf92480283ca4dd69f9d0a43&role.id=42c846f6944b40c8af5eeb8360cb7c7a => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:45.154281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02d2ee-747c-475a-8983-b289c776a33f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:45.154546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02d2ee-747c-475a-8983-b289c776a33f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:45.154815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02d2ee-747c-475a-8983-b289c776a33f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:45.156151 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 91/184] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:45.157207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528ab2-b647-4bca-a0e6-80d17fba4570 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:45.157486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528ab2-b647-4bca-a0e6-80d17fba4570 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:45.157762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528ab2-b647-4bca-a0e6-80d17fba4570 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:45.160015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 94/185] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:45.163215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 92/186] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:45.164210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f962b7d-1038-4328-9aa0-126185448c36 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:45.164538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f962b7d-1038-4328-9aa0-126185448c36 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:45.164819 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f962b7d-1038-4328-9aa0-126185448c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:45.165448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46528ab2-b647-4bca-a0e6-80d17fba4570 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:45.188621 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7f962b7d-1038-4328-9aa0-126185448c36 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:45.190543 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7f962b7d-1038-4328-9aa0-126185448c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:45.263163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:45.289197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 95/187] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:45.291570 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708c668-1859-4b7f-b484-42dfec8e405a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:45.292099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708c668-1859-4b7f-b484-42dfec8e405a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:45.292682 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708c668-1859-4b7f-b484-42dfec8e405a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:45.300532 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46528ab2-b647-4bca-a0e6-80d17fba4570 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:45.317430 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a708c668-1859-4b7f-b484-42dfec8e405a None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:45.319796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a708c668-1859-4b7f-b484-42dfec8e405a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:45.339585 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5845b-d5c5-4992-91d7-99b6e8c86ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:45.340243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5845b-d5c5-4992-91d7-99b6e8c86ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:45.340521 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5845b-d5c5-4992-91d7-99b6e8c86ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:45.340787 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5845b-d5c5-4992-91d7-99b6e8c86ab0 None admin] PATH_INFO: `/v3/users/90b72abbdbb34f7d9d5aa427e3749154` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:45.350378 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:45.357408 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5845b-d5c5-4992-91d7-99b6e8c86ab0 None admin] RBAC: Authorizing `identity:get_user(user_id=90b72abbdbb34f7d9d5aa427e3749154)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:45.358965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5845b-d5c5-4992-91d7-99b6e8c86ab0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:45.360622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 93/188] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:45 2022] GET /identity/v3/users/90b72abbdbb34f7d9d5aa427e3749154 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:45.362290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3e840-108c-470f-a614-4b0eabcbb566 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:45.362290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3e840-108c-470f-a614-4b0eabcbb566 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:45.362565 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3e840-108c-470f-a614-4b0eabcbb566 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:45.363876 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 94/189] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:45.366352 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3e840-108c-470f-a614-4b0eabcbb566 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:45.405114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 96/190] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:45.410495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708c668-1859-4b7f-b484-42dfec8e405a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:45.478010 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3e840-108c-470f-a614-4b0eabcbb566 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:45.505651 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708c668-1859-4b7f-b484-42dfec8e405a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:45.505833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:45.512935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c26ce0-32d1-4f9c-b5f1-2491dbf46e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:45.513622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c26ce0-32d1-4f9c-b5f1-2491dbf46e96 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:45.513838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c26ce0-32d1-4f9c-b5f1-2491dbf46e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:45.514107 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c26ce0-32d1-4f9c-b5f1-2491dbf46e96 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:45.524014 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c26ce0-32d1-4f9c-b5f1-2491dbf46e96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:45.528718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e5c26ce0-32d1-4f9c-b5f1-2491dbf46e96 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:12:45.528718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:45.528718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:45.528718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:45.528718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:45.528718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:45.528718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:45.528718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:45.528718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:45.528718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:45.528718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:45.528718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:45.528718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:45.528718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:45.528718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:45.528718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:45.529567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:45.529567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:45.529567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:45.529567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:45.529567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:45.529567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:45.529567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:45.529567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:45.529567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:45.529567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:45.529567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:45.529567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:45.529567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:45.529567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:45.529567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:45.529567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:45.530345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:45.530345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:45.530345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:45.530345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:45.530345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:45.530345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:45.530345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:45.530345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:45.530345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:45.530345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:45.530345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:12:45.530345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:45.530345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 95/191] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:45.531000 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059d9d4-3980-479f-be11-39910c0e07fd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:45.531222 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059d9d4-3980-479f-be11-39910c0e07fd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:45.531468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059d9d4-3980-479f-be11-39910c0e07fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:45.544659 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b01cf0-41f2-4569-b9a6-61a7fecfb03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:45.545292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b01cf0-41f2-4569-b9a6-61a7fecfb03d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:45.545573 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b01cf0-41f2-4569-b9a6-61a7fecfb03d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:45.545844 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b01cf0-41f2-4569-b9a6-61a7fecfb03d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:45.549267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e059d9d4-3980-479f-be11-39910c0e07fd None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:45.550472 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e059d9d4-3980-479f-be11-39910c0e07fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:45.576921 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:45.613058 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b01cf0-41f2-4569-b9a6-61a7fecfb03d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:45.614554 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b01cf0-41f2-4569-b9a6-61a7fecfb03d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:45.617352 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 96/192] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:45.622959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059d9d4-3980-479f-be11-39910c0e07fd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:45.712350 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 97/193] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:12:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:12:45.713186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b01cf0-41f2-4569-b9a6-61a7fecfb03d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:45.720890 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059d9d4-3980-479f-be11-39910c0e07fd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:45.741014 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:45.755870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899b12f-2fce-4a1c-98f4-013c0015d95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:45.756480 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899b12f-2fce-4a1c-98f4-013c0015d95d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:45.756760 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899b12f-2fce-4a1c-98f4-013c0015d95d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:45.757030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899b12f-2fce-4a1c-98f4-013c0015d95d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:45.758132 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899b12f-2fce-4a1c-98f4-013c0015d95d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:45.759626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899b12f-2fce-4a1c-98f4-013c0015d95d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:45.761330 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 97/194] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:12:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:45.761930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899b12f-2fce-4a1c-98f4-013c0015d95d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:45.779692 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:45.820226 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b01cf0-41f2-4569-b9a6-61a7fecfb03d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:45.853989 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d39ce-5dc8-4068-90c2-148ce9bf444f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:45.854612 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d39ce-5dc8-4068-90c2-148ce9bf444f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:45.854886 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d39ce-5dc8-4068-90c2-148ce9bf444f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:45.855156 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d39ce-5dc8-4068-90c2-148ce9bf444f None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:45.855817 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d39ce-5dc8-4068-90c2-148ce9bf444f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:45.856118 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d39ce-5dc8-4068-90c2-148ce9bf444f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:45.857481 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d39ce-5dc8-4068-90c2-148ce9bf444f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:45.861079 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899b12f-2fce-4a1c-98f4-013c0015d95d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:45.865844 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 98/195] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:12:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:45.866656 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d39ce-5dc8-4068-90c2-148ce9bf444f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:45.894317 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb0de1-0649-4042-b1e4-4380b391c5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:45.894939 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb0de1-0649-4042-b1e4-4380b391c5cc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:45.895210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb0de1-0649-4042-b1e4-4380b391c5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:45.895473 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb0de1-0649-4042-b1e4-4380b391c5cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:45.905711 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb0de1-0649-4042-b1e4-4380b391c5cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:45.910777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e7eb0de1-0649-4042-b1e4-4380b391c5cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:12:45.910777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:45.910777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:45.910777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:45.910777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:45.910777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:45.910777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:45.910777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:45.910777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:45.910777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:45.910777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:45.910777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:45.910777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:45.910777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:45.910777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:45.910777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:45.911633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:45.911633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:45.911633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:45.911633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:45.911633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:45.911633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:45.911633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:45.911633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:45.911633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:45.911633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:45.911633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:45.911633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:45.911633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:45.911633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:45.911633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:45.911633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:45.912990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:45.912990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:45.912990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:45.912990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:45.912990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:45.912990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:45.912990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:45.912990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:45.912990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:45.912990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:45.912990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:12:45.912990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:45.912990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 98/196] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:12:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:45.913642 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb0de1-0649-4042-b1e4-4380b391c5cc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:45.963140 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:45.970726 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d39ce-5dc8-4068-90c2-148ce9bf444f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:45.995431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:46.004765 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaab28-e7d5-45b3-8735-0e79afd48ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:46.005385 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaab28-e7d5-45b3-8735-0e79afd48ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:46.005660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaab28-e7d5-45b3-8735-0e79afd48ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:46.005930 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaab28-e7d5-45b3-8735-0e79afd48ab8 None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:46.007074 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeaab28-e7d5-45b3-8735-0e79afd48ab8 None admin] RBAC: Authorizing `identity:get_project(project_id=523fc40cf4044e598d36fb05a352d260)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:46.008494 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeaab28-e7d5-45b3-8735-0e79afd48ab8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:46.010104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 99/197] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:45 2022] GET /identity/v3/projects/523fc40cf4044e598d36fb05a352d260 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:46.010889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaab28-e7d5-45b3-8735-0e79afd48ab8 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:46.018601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb0de1-0649-4042-b1e4-4380b391c5cc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:46.051769 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818df97-a8dc-43ae-b413-9faaeccb279d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:46.052291 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818df97-a8dc-43ae-b413-9faaeccb279d None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:46.052572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818df97-a8dc-43ae-b413-9faaeccb279d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:46.052865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818df97-a8dc-43ae-b413-9faaeccb279d None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:46.053491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7818df97-a8dc-43ae-b413-9faaeccb279d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:46.054946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7818df97-a8dc-43ae-b413-9faaeccb279d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:46.076085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 99/198] 10.222.0.39 () {60 vars in 1132 bytes} [Mon Aug 8 02:12:45 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:12:46.076894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818df97-a8dc-43ae-b413-9faaeccb279d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:46.125944 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaab28-e7d5-45b3-8735-0e79afd48ab8 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:46.163641 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611e7ce-a689-4c82-a6cd-4b0ecbd18030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:46.164276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611e7ce-a689-4c82-a6cd-4b0ecbd18030 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:46.164551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611e7ce-a689-4c82-a6cd-4b0ecbd18030 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:46.164819 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611e7ce-a689-4c82-a6cd-4b0ecbd18030 None admin] PATH_INFO: `/v3/users/8ef3ff73ef554c22aa1dec9bab091558` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:46.165977 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611e7ce-a689-4c82-a6cd-4b0ecbd18030 None admin] RBAC: Authorizing `identity:get_user(user_id=8ef3ff73ef554c22aa1dec9bab091558)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:46.167496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611e7ce-a689-4c82-a6cd-4b0ecbd18030 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:46.169140 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 100/199] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:46 2022] GET /identity/v3/users/8ef3ff73ef554c22aa1dec9bab091558 => generated 289 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:46.169919 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611e7ce-a689-4c82-a6cd-4b0ecbd18030 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:46.194948 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818df97-a8dc-43ae-b413-9faaeccb279d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:46.197535 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:46.239224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a822c54-3862-43d7-a7fd-9c481fffcb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:46.239835 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a822c54-3862-43d7-a7fd-9c481fffcb7e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:46.240137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a822c54-3862-43d7-a7fd-9c481fffcb7e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:46.240403 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a822c54-3862-43d7-a7fd-9c481fffcb7e None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:46.240507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:46.241056 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a822c54-3862-43d7-a7fd-9c481fffcb7e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:46.241354 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a822c54-3862-43d7-a7fd-9c481fffcb7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:46.242938 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a822c54-3862-43d7-a7fd-9c481fffcb7e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:46.253034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 100/200] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:12:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:46.253881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a822c54-3862-43d7-a7fd-9c481fffcb7e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:46.290065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611e7ce-a689-4c82-a6cd-4b0ecbd18030 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:46.340254 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c4614-1b58-4259-ab1a-b8da6e8dfdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:46.340800 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c4614-1b58-4259-ab1a-b8da6e8dfdc1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:46.341075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c4614-1b58-4259-ab1a-b8da6e8dfdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:46.341490 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c4614-1b58-4259-ab1a-b8da6e8dfdc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:46.342297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c4614-1b58-4259-ab1a-b8da6e8dfdc1 None admin] RBAC: Adding query filter params (role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.project.id=523fc40cf4044e598d36fb05a352d260, user.id=90b72abbdbb34f7d9d5aa427e3749154) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:46.342662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c4614-1b58-4259-ab1a-b8da6e8dfdc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:46.345076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c4614-1b58-4259-ab1a-b8da6e8dfdc1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:46.352271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 101/201] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:12:46 2022] GET /identity/v3/role_assignments?user.id=90b72abbdbb34f7d9d5aa427e3749154&scope.project.id=523fc40cf4044e598d36fb05a352d260&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:46.353137 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c4614-1b58-4259-ab1a-b8da6e8dfdc1 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:46.397580 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a822c54-3862-43d7-a7fd-9c481fffcb7e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:46.437412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c424ba3-40df-4b32-86c0-6f02e27611c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:46.438041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c424ba3-40df-4b32-86c0-6f02e27611c6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:46.438336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c424ba3-40df-4b32-86c0-6f02e27611c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:46.438609 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c424ba3-40df-4b32-86c0-6f02e27611c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:46.477170 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c4614-1b58-4259-ab1a-b8da6e8dfdc1 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:46.481974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:46.508511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:46.522016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d09f3-4a3c-4304-b653-dce9fbcd7ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:46.522715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d09f3-4a3c-4304-b653-dce9fbcd7ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:46.522992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d09f3-4a3c-4304-b653-dce9fbcd7ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:46.523264 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d09f3-4a3c-4304-b653-dce9fbcd7ba0 None admin] PATH_INFO: `/v3/users/90b72abbdbb34f7d9d5aa427e3749154` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:46.524677 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d09f3-4a3c-4304-b653-dce9fbcd7ba0 None admin] RBAC: Authorizing `identity:get_user(user_id=90b72abbdbb34f7d9d5aa427e3749154)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:46.526178 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d09f3-4a3c-4304-b653-dce9fbcd7ba0 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:46.528082 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 102/202] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:46 2022] GET /identity/v3/users/90b72abbdbb34f7d9d5aa427e3749154 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:46.540526 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d09f3-4a3c-4304-b653-dce9fbcd7ba0 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:46.547532 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c424ba3-40df-4b32-86c0-6f02e27611c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:46.549318 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c424ba3-40df-4b32-86c0-6f02e27611c6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:46.662024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 101/203] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:12:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:12:46.667889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c424ba3-40df-4b32-86c0-6f02e27611c6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:46.689309 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d09f3-4a3c-4304-b653-dce9fbcd7ba0 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:46.760510 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:46.760845 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f96fc6-1772-41d3-81be-7309b5ebb8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:46.761664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f96fc6-1772-41d3-81be-7309b5ebb8bd None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:46.761664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f96fc6-1772-41d3-81be-7309b5ebb8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:46.761927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f96fc6-1772-41d3-81be-7309b5ebb8bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:46.828182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c424ba3-40df-4b32-86c0-6f02e27611c6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:46.842786 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:46.846640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f96fc6-1772-41d3-81be-7309b5ebb8bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:46.848238 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f96fc6-1772-41d3-81be-7309b5ebb8bd None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:46.909106 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d9ab0-9156-4ce5-9b68-cbe4fea8b1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:46.910541 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d9ab0-9156-4ce5-9b68-cbe4fea8b1ba None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:46.911025 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d9ab0-9156-4ce5-9b68-cbe4fea8b1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:46.911573 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d9ab0-9156-4ce5-9b68-cbe4fea8b1ba None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:46.916000 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d9ab0-9156-4ce5-9b68-cbe4fea8b1ba None admin] RBAC: Authorizing `identity:get_project(project_id=3441a04d36074741b13ebfa058e656b8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:46.917751 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d9ab0-9156-4ce5-9b68-cbe4fea8b1ba None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:46.919953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 102/204] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:46 2022] GET /identity/v3/projects/3441a04d36074741b13ebfa058e656b8 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:46.925394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d9ab0-9156-4ce5-9b68-cbe4fea8b1ba None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:46.956526 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 103/205] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:12:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 02:12:46.962032 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f96fc6-1772-41d3-81be-7309b5ebb8bd None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:47.124276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d9ab0-9156-4ce5-9b68-cbe4fea8b1ba None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:47.137654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:47.156446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f96fc6-1772-41d3-81be-7309b5ebb8bd None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:47.171469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a49ba-1b96-4ae2-8449-f11219c1babf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:47.172016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a49ba-1b96-4ae2-8449-f11219c1babf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:47.172296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a49ba-1b96-4ae2-8449-f11219c1babf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:47.172568 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a49ba-1b96-4ae2-8449-f11219c1babf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:47.173299 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a49ba-1b96-4ae2-8449-f11219c1babf None admin] RBAC: Adding query filter params (role.id=3b1d23e3e45d44a3a8069b8e823410b5, scope.project.id=3441a04d36074741b13ebfa058e656b8, user.id=8ef3ff73ef554c22aa1dec9bab091558) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:47.173606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a49ba-1b96-4ae2-8449-f11219c1babf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:47.175099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a49ba-1b96-4ae2-8449-f11219c1babf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:47.177053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:47.183299 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 103/206] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:12:46 2022] GET /identity/v3/role_assignments?user.id=8ef3ff73ef554c22aa1dec9bab091558&scope.project.id=3441a04d36074741b13ebfa058e656b8&role.id=3b1d23e3e45d44a3a8069b8e823410b5 => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:47.214067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e613786-d824-45d6-963d-b61bb433e7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:47.214736 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e613786-d824-45d6-963d-b61bb433e7ae None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:47.215012 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e613786-d824-45d6-963d-b61bb433e7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:47.215282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e613786-d824-45d6-963d-b61bb433e7ae None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:47.219047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e613786-d824-45d6-963d-b61bb433e7ae None admin] RBAC: Authorizing `identity:get_project(project_id=523fc40cf4044e598d36fb05a352d260)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:47.220512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e613786-d824-45d6-963d-b61bb433e7ae None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:47.222616 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 104/207] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:46 2022] GET /identity/v3/projects/523fc40cf4044e598d36fb05a352d260 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:47.228841 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a49ba-1b96-4ae2-8449-f11219c1babf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:47.346041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a49ba-1b96-4ae2-8449-f11219c1babf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:47.384946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa3b50-5332-4975-9901-4351a1cb52e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:47.385560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa3b50-5332-4975-9901-4351a1cb52e7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:47.385839 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa3b50-5332-4975-9901-4351a1cb52e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:47.386107 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa3b50-5332-4975-9901-4351a1cb52e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:47.386867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa3b50-5332-4975-9901-4351a1cb52e7 None admin] RBAC: Adding query filter params (role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839, scope.project.id=523fc40cf4044e598d36fb05a352d260, user.id=90b72abbdbb34f7d9d5aa427e3749154) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:47.387177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa3b50-5332-4975-9901-4351a1cb52e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:47.388611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa3b50-5332-4975-9901-4351a1cb52e7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:47.394571 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 104/208] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:12:47 2022] GET /identity/v3/role_assignments?user.id=90b72abbdbb34f7d9d5aa427e3749154&scope.project.id=523fc40cf4044e598d36fb05a352d260&role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:48.494912 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acddaa3-2062-45b5-8b2e-b2fbde20db22 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:48.495874 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acddaa3-2062-45b5-8b2e-b2fbde20db22 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:48.496339 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acddaa3-2062-45b5-8b2e-b2fbde20db22 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:48.497899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 105/209] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:48.503452 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4402df-77e2-48af-867e-6fd9f0fd715f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:48.504289 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4402df-77e2-48af-867e-6fd9f0fd715f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:48.504753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4402df-77e2-48af-867e-6fd9f0fd715f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:48.524402 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-df4402df-77e2-48af-867e-6fd9f0fd715f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:48.525613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-df4402df-77e2-48af-867e-6fd9f0fd715f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:48.580020 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3696f-9d8d-4956-b9b9-3f71d58c6b79 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:48.580601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3696f-9d8d-4956-b9b9-3f71d58c6b79 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:48.581069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3696f-9d8d-4956-b9b9-3f71d58c6b79 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:48.582602 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 106/210] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:48.583940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615e061-c57c-4144-a8af-65bd34cab359 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:48.584441 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615e061-c57c-4144-a8af-65bd34cab359 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:48.584927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615e061-c57c-4144-a8af-65bd34cab359 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:48.586608 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 107/211] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:48.587878 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbc5b8-3c67-47bf-b603-bf8d9ded3d67 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:48.588160 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbc5b8-3c67-47bf-b603-bf8d9ded3d67 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:48.588429 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbc5b8-3c67-47bf-b603-bf8d9ded3d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:48.593086 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:48.605925 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 105/212] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:48.606376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3558ab-08ec-40ef-a211-fc69bbde79a1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:48.606666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3558ab-08ec-40ef-a211-fc69bbde79a1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:48.606942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3558ab-08ec-40ef-a211-fc69bbde79a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:48.616102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-facbc5b8-3c67-47bf-b603-bf8d9ded3d67 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:48.617334 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-facbc5b8-3c67-47bf-b603-bf8d9ded3d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:48.627349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3a3558ab-08ec-40ef-a211-fc69bbde79a1 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:48.628735 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3a3558ab-08ec-40ef-a211-fc69bbde79a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:48.691730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 108/213] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:48.692399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbc5b8-3c67-47bf-b603-bf8d9ded3d67 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:48.702787 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 106/214] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:48.716168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3558ab-08ec-40ef-a211-fc69bbde79a1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:48.793898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:48.804959 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbc5b8-3c67-47bf-b603-bf8d9ded3d67 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:48.826754 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3558ab-08ec-40ef-a211-fc69bbde79a1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:48.844658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d52ba-1808-4221-8dad-ba568f6d7f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:48.845355 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d52ba-1808-4221-8dad-ba568f6d7f86 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:48.845630 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d52ba-1808-4221-8dad-ba568f6d7f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:48.845896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d52ba-1808-4221-8dad-ba568f6d7f86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:48.851254 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:48.857062 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d52ba-1808-4221-8dad-ba568f6d7f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:48.861794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-cb8d52ba-1808-4221-8dad-ba568f6d7f86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:48.861794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:48.861794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:48.861794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:48.861794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:48.861794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:48.861794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:48.861794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:48.861794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:48.861794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:48.861794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:48.861794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:48.861794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:48.861794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:48.861794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:48.861794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:48.862714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:48.862714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:48.862714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:48.862714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:48.862714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:48.862714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:48.862714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:48.862714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:48.862714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:48.862714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:48.862714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:48.862714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:48.862714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:48.862714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:48.862714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:48.862714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:48.863488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:48.863488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:48.863488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:48.863488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:48.863488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:48.863488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:48.863488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:48.863488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:48.863488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:48.863488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:48.863488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:48.863488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:12:48.863488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 109/215] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:48.864117 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d52ba-1808-4221-8dad-ba568f6d7f86 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:48.870584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac181b-2bb3-40aa-a8d0-f7c3f9bd57ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:48.871227 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac181b-2bb3-40aa-a8d0-f7c3f9bd57ed None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:48.871505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac181b-2bb3-40aa-a8d0-f7c3f9bd57ed None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:48.871773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac181b-2bb3-40aa-a8d0-f7c3f9bd57ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:48.872930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac181b-2bb3-40aa-a8d0-f7c3f9bd57ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:48.874425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac181b-2bb3-40aa-a8d0-f7c3f9bd57ed None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:48.876006 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 107/216] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:12:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:48.879417 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac181b-2bb3-40aa-a8d0-f7c3f9bd57ed None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:48.965341 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d52ba-1808-4221-8dad-ba568f6d7f86 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:48.991544 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac181b-2bb3-40aa-a8d0-f7c3f9bd57ed None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:49.003972 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5349b3d-7199-41ab-b74c-35f260ba38a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:49.004601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5349b3d-7199-41ab-b74c-35f260ba38a9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:49.004883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5349b3d-7199-41ab-b74c-35f260ba38a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:49.005168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5349b3d-7199-41ab-b74c-35f260ba38a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:49.016369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5349b3d-7199-41ab-b74c-35f260ba38a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:49.021231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-f5349b3d-7199-41ab-b74c-35f260ba38a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:12:49.021231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:49.021231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:49.021231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:49.021231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:49.021231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:49.021231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:49.021231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:49.021231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:49.021231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:49.021231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:49.021231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:49.021231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:49.021231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:49.021231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:49.021231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:49.022603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:49.022603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:49.022603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:49.022603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:49.022603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:49.022603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:49.022603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:49.022603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:49.022603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:49.022603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:49.022603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:49.022603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:49.022603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:49.022603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:49.022603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:49.022603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:49.023488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:49.023488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:49.023488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:49.023488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:49.023488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:49.023488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:49.023488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:49.023488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:49.023488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:49.023488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:49.023488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:12:49.023488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:12:49.023488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 110/217] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:12:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:49.023488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5349b3d-7199-41ab-b74c-35f260ba38a9 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:49.029788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6f0b7-1f1c-4a1c-8c5c-b5bb5175c5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:49.030436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6f0b7-1f1c-4a1c-8c5c-b5bb5175c5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:49.030710 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6f0b7-1f1c-4a1c-8c5c-b5bb5175c5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:49.030971 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6f0b7-1f1c-4a1c-8c5c-b5bb5175c5d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:49.031621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6f0b7-1f1c-4a1c-8c5c-b5bb5175c5d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:49.031917 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6f0b7-1f1c-4a1c-8c5c-b5bb5175c5d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:49.033289 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6f0b7-1f1c-4a1c-8c5c-b5bb5175c5d5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:49.042149 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:49.042389 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 108/218] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:12:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:49.047829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6f0b7-1f1c-4a1c-8c5c-b5bb5175c5d5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:49.106899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:49.133096 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5349b3d-7199-41ab-b74c-35f260ba38a9 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:49.161026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6f0b7-1f1c-4a1c-8c5c-b5bb5175c5d5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:49.169884 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df0bc2-96a1-46c0-9cc7-e8e4a7a7b5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:49.170527 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df0bc2-96a1-46c0-9cc7-e8e4a7a7b5e2 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:49.170808 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df0bc2-96a1-46c0-9cc7-e8e4a7a7b5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:49.171083 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df0bc2-96a1-46c0-9cc7-e8e4a7a7b5e2 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:49.171734 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df0bc2-96a1-46c0-9cc7-e8e4a7a7b5e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:49.173915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df0bc2-96a1-46c0-9cc7-e8e4a7a7b5e2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:49.200855 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80489630-cece-42b9-9a34-5350d9ea141e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:49.210652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80489630-cece-42b9-9a34-5350d9ea141e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:49.210874 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 111/219] 10.222.0.39 () {60 vars in 1132 bytes} [Mon Aug 8 02:12:49 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:12:49.210968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80489630-cece-42b9-9a34-5350d9ea141e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:49.210968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80489630-cece-42b9-9a34-5350d9ea141e None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:49.210968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80489630-cece-42b9-9a34-5350d9ea141e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:49.211178 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df0bc2-96a1-46c0-9cc7-e8e4a7a7b5e2 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:49.211255 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80489630-cece-42b9-9a34-5350d9ea141e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:49.211255 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80489630-cece-42b9-9a34-5350d9ea141e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:49.225551 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 109/220] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:49.226603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80489630-cece-42b9-9a34-5350d9ea141e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:49.308915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:49.361640 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80489630-cece-42b9-9a34-5350d9ea141e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:49.363531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df0bc2-96a1-46c0-9cc7-e8e4a7a7b5e2 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:49.378074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:49.403630 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7f22c-75fb-4c6b-b7d0-d04528eb81dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:49.404276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7f22c-75fb-4c6b-b7d0-d04528eb81dc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:49.404560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7f22c-75fb-4c6b-b7d0-d04528eb81dc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:49.404851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7f22c-75fb-4c6b-b7d0-d04528eb81dc None admin] PATH_INFO: `/v3/users/90b72abbdbb34f7d9d5aa427e3749154` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:49.405994 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7f22c-75fb-4c6b-b7d0-d04528eb81dc None admin] RBAC: Authorizing `identity:get_user(user_id=90b72abbdbb34f7d9d5aa427e3749154)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:49.407607 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7f22c-75fb-4c6b-b7d0-d04528eb81dc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:49.409518 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 110/221] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:49 2022] GET /identity/v3/users/90b72abbdbb34f7d9d5aa427e3749154 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:49.410039 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0f42d-d802-4090-9921-205232285f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:49.410965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0f42d-d802-4090-9921-205232285f24 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:49.411245 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0f42d-d802-4090-9921-205232285f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:49.411514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0f42d-d802-4090-9921-205232285f24 None admin] PATH_INFO: `/v3/users/1551c4b972544523bdf81960f5e512d9` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:49.415209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7f22c-75fb-4c6b-b7d0-d04528eb81dc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:49.433520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0f42d-d802-4090-9921-205232285f24 None admin] RBAC: Authorizing `identity:get_user(user_id=1551c4b972544523bdf81960f5e512d9)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:49.436359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0f42d-d802-4090-9921-205232285f24 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:49.438645 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 112/222] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:49 2022] GET /identity/v3/users/1551c4b972544523bdf81960f5e512d9 => generated 297 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:49.446951 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0f42d-d802-4090-9921-205232285f24 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:49.542074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7f22c-75fb-4c6b-b7d0-d04528eb81dc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:49.572151 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0f42d-d802-4090-9921-205232285f24 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:49.581747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8b383-93ff-41cb-9efd-5478e64e3a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:49.582779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8b383-93ff-41cb-9efd-5478e64e3a95 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:49.582779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8b383-93ff-41cb-9efd-5478e64e3a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:49.583033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8b383-93ff-41cb-9efd-5478e64e3a95 None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:49.584972 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8b383-93ff-41cb-9efd-5478e64e3a95 None admin] RBAC: Authorizing `identity:get_project(project_id=523fc40cf4044e598d36fb05a352d260)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:49.586169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8b383-93ff-41cb-9efd-5478e64e3a95 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:49.587874 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 111/223] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:49 2022] GET /identity/v3/projects/523fc40cf4044e598d36fb05a352d260 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:49.589053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097a135-a6d8-4878-80a0-eef369d6ebde None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:49.589293 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097a135-a6d8-4878-80a0-eef369d6ebde None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:49.589566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097a135-a6d8-4878-80a0-eef369d6ebde None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:49.592033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 112/224] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:49.595097 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097a135-a6d8-4878-80a0-eef369d6ebde None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:49.608438 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:49.621706 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c5ab4-e0f3-4723-9c57-454f48ed73e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:49.622359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c5ab4-e0f3-4723-9c57-454f48ed73e4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:49.622733 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c5ab4-e0f3-4723-9c57-454f48ed73e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:49.622905 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c5ab4-e0f3-4723-9c57-454f48ed73e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:49.656449 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:49.697575 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c5ab4-e0f3-4723-9c57-454f48ed73e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:49.699096 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c5ab4-e0f3-4723-9c57-454f48ed73e4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:49.720461 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097a135-a6d8-4878-80a0-eef369d6ebde None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:49.766668 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4b68c-4594-4ad0-a98b-83f2e5498c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:49.767354 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4b68c-4594-4ad0-a98b-83f2e5498c53 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:49.767627 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4b68c-4594-4ad0-a98b-83f2e5498c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:49.768210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4b68c-4594-4ad0-a98b-83f2e5498c53 None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260/users/90b72abbdbb34f7d9d5aa427e3749154/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:49.770942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4b68c-4594-4ad0-a98b-83f2e5498c53 None admin] RBAC: Authorizing `identity:create_grant(project_id=523fc40cf4044e598d36fb05a352d260, user_id=90b72abbdbb34f7d9d5aa427e3749154, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:49.773596 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4b68c-4594-4ad0-a98b-83f2e5498c53 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:49.791097 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 113/225] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:12:49 2022] PUT /identity/v3/projects/523fc40cf4044e598d36fb05a352d260/users/90b72abbdbb34f7d9d5aa427e3749154/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:12:49.792500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b65b6-6681-47d6-b9b7-62b61a58b708 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:49.792725 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b65b6-6681-47d6-b9b7-62b61a58b708 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:49.793289 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b65b6-6681-47d6-b9b7-62b61a58b708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:49.817420 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-623b65b6-6681-47d6-b9b7-62b61a58b708 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:49.819544 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-623b65b6-6681-47d6-b9b7-62b61a58b708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:49.837579 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 113/226] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:12:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:12:49.839921 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c5ab4-e0f3-4723-9c57-454f48ed73e4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:49.886949 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:49.960379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 114/227] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:49.991437 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b65b6-6681-47d6-b9b7-62b61a58b708 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:50.005136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c5ab4-e0f3-4723-9c57-454f48ed73e4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:50.045048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:50.049752 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49413a2-a48a-4723-9058-2535501ae221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:50.050547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49413a2-a48a-4723-9058-2535501ae221 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:50.050930 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49413a2-a48a-4723-9058-2535501ae221 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:50.051346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49413a2-a48a-4723-9058-2535501ae221 None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:50.053126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49413a2-a48a-4723-9058-2535501ae221 None admin] RBAC: Authorizing `identity:get_project(project_id=523fc40cf4044e598d36fb05a352d260)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:50.055077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49413a2-a48a-4723-9058-2535501ae221 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:50.056891 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 114/228] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:49 2022] GET /identity/v3/projects/523fc40cf4044e598d36fb05a352d260 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:50.063507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49413a2-a48a-4723-9058-2535501ae221 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:50.143504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b65b6-6681-47d6-b9b7-62b61a58b708 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:50.174979 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:50.183383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49413a2-a48a-4723-9058-2535501ae221 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:50.201816 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d7663-bfc4-428f-a512-0047504ac573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:50.207093 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d7663-bfc4-428f-a512-0047504ac573 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:50.207093 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d7663-bfc4-428f-a512-0047504ac573 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:50.207420 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d7663-bfc4-428f-a512-0047504ac573 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:50.222724 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af607764-520c-4e96-a2aa-ef24a53eeb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:50.226341 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af607764-520c-4e96-a2aa-ef24a53eeb56 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:50.227056 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af607764-520c-4e96-a2aa-ef24a53eeb56 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:50.227817 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d7663-bfc4-428f-a512-0047504ac573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:50.229725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af607764-520c-4e96-a2aa-ef24a53eeb56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:50.230708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af607764-520c-4e96-a2aa-ef24a53eeb56 None admin] RBAC: Adding query filter params (role.id=42c846f6944b40c8af5eeb8360cb7c7a, scope.project.id=523fc40cf4044e598d36fb05a352d260, user.id=1551c4b972544523bdf81960f5e512d9) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:50.232396 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af607764-520c-4e96-a2aa-ef24a53eeb56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:50.233984 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af607764-520c-4e96-a2aa-ef24a53eeb56 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:50.238332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ee9d7663-bfc4-428f-a512-0047504ac573 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:12:50.238332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:50.238332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:50.238332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:50.238332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:50.238332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:50.238332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:50.238332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:50.238332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:50.238332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:50.238332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:50.238332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:50.238332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:50.238332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:50.238332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:50.238332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:50.239443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:50.239443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:50.239443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:50.239443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:50.239443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:50.239443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:50.239443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:50.239443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:50.239443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:50.239443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:50.239443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:50.239443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:50.239443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:50.239443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:50.239443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:50.239443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:50.240941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:50.240941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:50.240941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:50.240941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:50.240941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:50.240941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:50.240941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:50.240941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:50.240941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:50.240941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:50.240941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:12:50.240941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:50.240941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 115/229] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:12:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:12:50.246432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 115/230] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:12:50 2022] GET /identity/v3/role_assignments?user.id=1551c4b972544523bdf81960f5e512d9&scope.project.id=523fc40cf4044e598d36fb05a352d260&role.id=42c846f6944b40c8af5eeb8360cb7c7a => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:50.250399 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d7663-bfc4-428f-a512-0047504ac573 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:50.412650 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:50.422277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d7663-bfc4-428f-a512-0047504ac573 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:50.457652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e42bbb-bf75-4793-b6ad-cb232c2a67f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:50.458360 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e42bbb-bf75-4793-b6ad-cb232c2a67f8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:50.458652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e42bbb-bf75-4793-b6ad-cb232c2a67f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:50.459489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e42bbb-bf75-4793-b6ad-cb232c2a67f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:50.460158 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e42bbb-bf75-4793-b6ad-cb232c2a67f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:50.460463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e42bbb-bf75-4793-b6ad-cb232c2a67f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:50.461855 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e42bbb-bf75-4793-b6ad-cb232c2a67f8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:50.470514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 116/231] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:12:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:50.476657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af607764-520c-4e96-a2aa-ef24a53eeb56 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:50.626010 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af607764-520c-4e96-a2aa-ef24a53eeb56 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:50.663975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c392d6f-2fba-42ad-aa88-b0d8108a9dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:50.664929 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c392d6f-2fba-42ad-aa88-b0d8108a9dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:50.665430 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c392d6f-2fba-42ad-aa88-b0d8108a9dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:50.665959 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c392d6f-2fba-42ad-aa88-b0d8108a9dc3 None admin] PATH_INFO: `/v3/users/90b72abbdbb34f7d9d5aa427e3749154` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:50.667573 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c392d6f-2fba-42ad-aa88-b0d8108a9dc3 None admin] RBAC: Authorizing `identity:get_user(user_id=90b72abbdbb34f7d9d5aa427e3749154)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:50.669250 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c392d6f-2fba-42ad-aa88-b0d8108a9dc3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:50.671209 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 116/232] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:50 2022] GET /identity/v3/users/90b72abbdbb34f7d9d5aa427e3749154 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:50.676804 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e42bbb-bf75-4793-b6ad-cb232c2a67f8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:50.817649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e42bbb-bf75-4793-b6ad-cb232c2a67f8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:50.852290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98c931-7e19-4f11-8d0c-2b8c39205ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:50.852919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98c931-7e19-4f11-8d0c-2b8c39205ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:50.853196 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98c931-7e19-4f11-8d0c-2b8c39205ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:50.853501 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98c931-7e19-4f11-8d0c-2b8c39205ff5 None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:50.854631 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98c931-7e19-4f11-8d0c-2b8c39205ff5 None admin] RBAC: Authorizing `identity:get_project(project_id=523fc40cf4044e598d36fb05a352d260)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:50.856084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98c931-7e19-4f11-8d0c-2b8c39205ff5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:50.857721 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 117/233] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:50 2022] GET /identity/v3/projects/523fc40cf4044e598d36fb05a352d260 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:50.863672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c392d6f-2fba-42ad-aa88-b0d8108a9dc3 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:50.907417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:50.975091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c392d6f-2fba-42ad-aa88-b0d8108a9dc3 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:51.009365 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544bc34-c17f-4855-82eb-c3622b4648e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:51.010077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544bc34-c17f-4855-82eb-c3622b4648e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:51.010389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544bc34-c17f-4855-82eb-c3622b4648e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:51.010666 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544bc34-c17f-4855-82eb-c3622b4648e9 None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260/users/90b72abbdbb34f7d9d5aa427e3749154/roles/9f8a5b177f4a41fc8bc88ff4f1ce0839` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:51.012837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544bc34-c17f-4855-82eb-c3622b4648e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=523fc40cf4044e598d36fb05a352d260, user_id=90b72abbdbb34f7d9d5aa427e3749154, role_id=9f8a5b177f4a41fc8bc88ff4f1ce0839)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:51.014310 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544bc34-c17f-4855-82eb-c3622b4648e9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:51.032893 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 117/234] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:12:50 2022] PUT /identity/v3/projects/523fc40cf4044e598d36fb05a352d260/users/90b72abbdbb34f7d9d5aa427e3749154/roles/9f8a5b177f4a41fc8bc88ff4f1ce0839 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:12:51.849244 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a49f9a-30e8-495b-9748-78658f934244 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:51.850283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a49f9a-30e8-495b-9748-78658f934244 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:51.850362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a49f9a-30e8-495b-9748-78658f934244 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:51.851929 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 118/235] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:51.856982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c0318-1a90-4aa9-9f75-bc856de5694c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:51.857369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c0318-1a90-4aa9-9f75-bc856de5694c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:51.857529 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c0318-1a90-4aa9-9f75-bc856de5694c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:51.876217 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d34c0318-1a90-4aa9-9f75-bc856de5694c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:51.877429 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d34c0318-1a90-4aa9-9f75-bc856de5694c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:51.963059 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 118/236] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:51.997533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a49f9a-30e8-495b-9748-78658f934244 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:52.019976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447ff20-5bb6-4996-9b8e-d876fd561a87 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:52.020288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447ff20-5bb6-4996-9b8e-d876fd561a87 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:52.020530 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447ff20-5bb6-4996-9b8e-d876fd561a87 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:52.022715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 119/237] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:52.038492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:52.039522 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcadd9a-392a-4cc6-962c-47d969468ff1 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:52.039804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcadd9a-392a-4cc6-962c-47d969468ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:52.040146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcadd9a-392a-4cc6-962c-47d969468ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:52.060565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-adcadd9a-392a-4cc6-962c-47d969468ff1 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:52.061790 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-adcadd9a-392a-4cc6-962c-47d969468ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:52.078606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:52.161620 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a49f9a-30e8-495b-9748-78658f934244 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:52.183335 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 120/238] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 08 02:12:52.204590 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22283f35-5eaa-4363-b30f-24ef367c45f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:52.205212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22283f35-5eaa-4363-b30f-24ef367c45f3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:52.205484 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22283f35-5eaa-4363-b30f-24ef367c45f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:52.205747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22283f35-5eaa-4363-b30f-24ef367c45f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:52.206911 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22283f35-5eaa-4363-b30f-24ef367c45f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:52.208412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22283f35-5eaa-4363-b30f-24ef367c45f3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:52.211001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 119/239] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:12:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:52.216713 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcadd9a-392a-4cc6-962c-47d969468ff1 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:52.232618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22283f35-5eaa-4363-b30f-24ef367c45f3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:52.338399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcadd9a-392a-4cc6-962c-47d969468ff1 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:52.366669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:52.370491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22283f35-5eaa-4363-b30f-24ef367c45f3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:52.375122 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae980f-2294-4f5e-96c7-c9949d9d861e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:52.375896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae980f-2294-4f5e-96c7-c9949d9d861e None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:52.376313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae980f-2294-4f5e-96c7-c9949d9d861e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:52.376721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae980f-2294-4f5e-96c7-c9949d9d861e None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:52.377526 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae980f-2294-4f5e-96c7-c9949d9d861e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:52.379310 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae980f-2294-4f5e-96c7-c9949d9d861e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:52.404041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 121/240] 10.222.0.39 () {60 vars in 1132 bytes} [Mon Aug 8 02:12:52 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:12:52.405740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1ede2-1a47-4c8b-80f1-b46711e2e427 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:52.406190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1ede2-1a47-4c8b-80f1-b46711e2e427 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:52.406761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1ede2-1a47-4c8b-80f1-b46711e2e427 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:52.408300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 122/241] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:52.416400 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dff3c-8052-49ff-8084-78e54a0cf5f3 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:52.416992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dff3c-8052-49ff-8084-78e54a0cf5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:52.417440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dff3c-8052-49ff-8084-78e54a0cf5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:52.420772 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92fa5b-9c9c-4482-883b-cc98c902d6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:52.422056 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92fa5b-9c9c-4482-883b-cc98c902d6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:52.422480 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92fa5b-9c9c-4482-883b-cc98c902d6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:52.422973 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92fa5b-9c9c-4482-883b-cc98c902d6d1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:52.427855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:52.437040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92fa5b-9c9c-4482-883b-cc98c902d6d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:52.449333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3b92fa5b-9c9c-4482-883b-cc98c902d6d1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:12:52.449333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:52.449333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:52.449333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:52.449333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:52.449333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:52.449333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:52.449333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:52.449333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:52.449333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:52.449333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:52.449333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:52.449333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:52.449333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:52.449333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:52.449333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:52.450215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:52.450215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:52.450215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:52.450215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:52.450215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:52.450215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:52.450215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:52.450215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:52.450215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:52.450215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:52.450215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:52.450215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:52.450215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:52.450215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:52.450215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:52.450215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:52.451060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:52.451060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:52.451060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:52.451060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:52.451060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:52.451060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:52.451060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:52.451060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:52.451060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:52.451060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:52.451060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:12:52.451060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:52.453677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 120/242] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:12:52.459914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d67dff3c-8052-49ff-8084-78e54a0cf5f3 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:52.461564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d67dff3c-8052-49ff-8084-78e54a0cf5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:52.465307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92fa5b-9c9c-4482-883b-cc98c902d6d1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:52.612431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 123/243] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:52.621339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92fa5b-9c9c-4482-883b-cc98c902d6d1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:52.652324 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dff3c-8052-49ff-8084-78e54a0cf5f3 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:52.658415 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2481f-1e85-466b-8bbf-343bebd5a2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:52.659316 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2481f-1e85-466b-8bbf-343bebd5a2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:52.659796 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2481f-1e85-466b-8bbf-343bebd5a2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:52.660265 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2481f-1e85-466b-8bbf-343bebd5a2e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:52.661144 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2481f-1e85-466b-8bbf-343bebd5a2e8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:52.661661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2481f-1e85-466b-8bbf-343bebd5a2e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:52.663379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2481f-1e85-466b-8bbf-343bebd5a2e8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:52.672411 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 121/244] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:12:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:52.679442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2481f-1e85-466b-8bbf-343bebd5a2e8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:52.710632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:52.794832 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dff3c-8052-49ff-8084-78e54a0cf5f3 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:52.797468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2481f-1e85-466b-8bbf-343bebd5a2e8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:52.839551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:52.842412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc4946-bdbe-4c15-8b6d-3901d6f8adc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:52.843301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc4946-bdbe-4c15-8b6d-3901d6f8adc7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:52.844668 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc4946-bdbe-4c15-8b6d-3901d6f8adc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:52.845039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc4946-bdbe-4c15-8b6d-3901d6f8adc7 None admin] PATH_INFO: `/v3/users/6ec6b5f76e9c4f16a5565ea1629b3563` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:52.846794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ae9e9-5198-4719-b7f8-ddf7afb5d5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:52.847423 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ae9e9-5198-4719-b7f8-ddf7afb5d5fc None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:52.847758 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ae9e9-5198-4719-b7f8-ddf7afb5d5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:52.848039 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ae9e9-5198-4719-b7f8-ddf7afb5d5fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:52.861517 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ae9e9-5198-4719-b7f8-ddf7afb5d5fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:52.862106 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc4946-bdbe-4c15-8b6d-3901d6f8adc7 None admin] RBAC: Authorizing `identity:get_user(user_id=6ec6b5f76e9c4f16a5565ea1629b3563)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:52.863545 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc4946-bdbe-4c15-8b6d-3901d6f8adc7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:52.866229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 122/245] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:52 2022] GET /identity/v3/users/6ec6b5f76e9c4f16a5565ea1629b3563 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:52.867475 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413be8c-da76-480a-93f3-06d6e471eec8 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:52.867755 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413be8c-da76-480a-93f3-06d6e471eec8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:52.868026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413be8c-da76-480a-93f3-06d6e471eec8 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:52.869252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 123/246] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:52.871728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-581ae9e9-5198-4719-b7f8-ddf7afb5d5fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:12:52.871728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:52.871728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:52.871728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:52.871728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:52.871728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:52.871728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:52.871728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:52.871728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:52.871728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:52.871728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:52.871728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:52.871728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:52.871728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:52.871728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:52.871728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:52.872580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:52.872580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:52.872580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:52.872580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:52.872580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:52.872580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:52.872580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:52.872580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:52.872580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:52.872580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:52.872580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:52.872580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:52.872580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:52.872580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:52.872580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:52.872580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:52.873346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:52.873346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:52.873346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:52.873346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:52.873346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:52.873346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:52.873346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:52.873346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:52.873346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:52.873346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:52.873346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:12:52.873346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:12:52.874490 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 124/247] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:12:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:12:52.875645 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413be8c-da76-480a-93f3-06d6e471eec8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:52.876630 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac203c1b-8725-471e-bdae-1006d347041c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:52.876913 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac203c1b-8725-471e-bdae-1006d347041c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:52.877185 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac203c1b-8725-471e-bdae-1006d347041c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:52.898940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ac203c1b-8725-471e-bdae-1006d347041c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:52.900100 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ac203c1b-8725-471e-bdae-1006d347041c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:52.975204 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:52.995657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 125/248] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:53.002788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac203c1b-8725-471e-bdae-1006d347041c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:53.028107 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413be8c-da76-480a-93f3-06d6e471eec8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:53.089826 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae11602-3dbc-4c89-84cd-42d53d2392db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:53.090939 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae11602-3dbc-4c89-84cd-42d53d2392db None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:53.091422 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae11602-3dbc-4c89-84cd-42d53d2392db None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:53.091928 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae11602-3dbc-4c89-84cd-42d53d2392db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:53.165851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae11602-3dbc-4c89-84cd-42d53d2392db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:53.167931 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae11602-3dbc-4c89-84cd-42d53d2392db None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:53.257185 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac203c1b-8725-471e-bdae-1006d347041c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:53.278599 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:53.294560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:53.314186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7a57d-423b-4d0c-8a30-8c98d7353b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:53.315212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7a57d-423b-4d0c-8a30-8c98d7353b08 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:53.315641 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7a57d-423b-4d0c-8a30-8c98d7353b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:53.316060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7a57d-423b-4d0c-8a30-8c98d7353b08 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:53.317126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7a57d-423b-4d0c-8a30-8c98d7353b08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:53.317570 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7a57d-423b-4d0c-8a30-8c98d7353b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:53.319636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7a57d-423b-4d0c-8a30-8c98d7353b08 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:53.330185 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 126/249] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:53.331466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7a57d-423b-4d0c-8a30-8c98d7353b08 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:53.333745 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 124/250] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:12:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:12:53.336215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae11602-3dbc-4c89-84cd-42d53d2392db None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:53.454471 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae11602-3dbc-4c89-84cd-42d53d2392db None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:53.475879 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7a57d-423b-4d0c-8a30-8c98d7353b08 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:53.509450 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83684396-9cf4-4fc7-a2dd-e5ae3a86884a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:53.509832 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efec835-368e-447c-b690-35d5da7568ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:53.510044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83684396-9cf4-4fc7-a2dd-e5ae3a86884a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:53.510297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83684396-9cf4-4fc7-a2dd-e5ae3a86884a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:53.510508 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efec835-368e-447c-b690-35d5da7568ac None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:53.510609 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83684396-9cf4-4fc7-a2dd-e5ae3a86884a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:53.510786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efec835-368e-447c-b690-35d5da7568ac None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:53.511057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efec835-368e-447c-b690-35d5da7568ac None admin] PATH_INFO: `/v3/users/90b72abbdbb34f7d9d5aa427e3749154` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:53.512337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efec835-368e-447c-b690-35d5da7568ac None admin] RBAC: Authorizing `identity:get_user(user_id=90b72abbdbb34f7d9d5aa427e3749154)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:53.513962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efec835-368e-447c-b690-35d5da7568ac None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:53.515889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 125/251] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:53 2022] GET /identity/v3/users/90b72abbdbb34f7d9d5aa427e3749154 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:53.516659 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efec835-368e-447c-b690-35d5da7568ac None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:53.521911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83684396-9cf4-4fc7-a2dd-e5ae3a86884a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:53.526642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-83684396-9cf4-4fc7-a2dd-e5ae3a86884a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:53.526642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:53.526642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:53.526642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:53.526642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:53.526642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:53.526642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:53.526642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:53.526642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:53.526642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:53.526642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:53.526642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:53.526642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:53.526642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:53.526642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:53.526642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:53.527499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:53.527499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:53.527499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:53.527499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:53.527499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:53.527499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:53.527499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:53.527499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:53.527499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:53.527499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:53.527499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:53.527499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:53.527499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:53.527499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:53.527499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:53.527499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:53.528278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:53.528278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:53.528278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:53.528278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:53.528278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:53.528278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:53.528278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:53.528278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:53.528278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:53.528278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:53.528278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:53.528278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:12:53.528278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 127/252] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:53.529519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc794a-b671-42ec-ba01-409d1ff87d03 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:53.529678 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc794a-b671-42ec-ba01-409d1ff87d03 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:53.531240 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc794a-b671-42ec-ba01-409d1ff87d03 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:53.531240 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 128/253] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:53.531843 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc794a-b671-42ec-ba01-409d1ff87d03 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:53.577226 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:53.579765 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:53.636510 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc794a-b671-42ec-ba01-409d1ff87d03 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:53.637782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efec835-368e-447c-b690-35d5da7568ac None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:53.671966 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135974c-6d73-4e20-8326-e1cb7847e3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:53.672567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135974c-6d73-4e20-8326-e1cb7847e3ad None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:53.672842 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135974c-6d73-4e20-8326-e1cb7847e3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:53.673124 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135974c-6d73-4e20-8326-e1cb7847e3ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:53.676413 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf30edd-39fd-4c13-88e6-f60893c605a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:53.677307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf30edd-39fd-4c13-88e6-f60893c605a5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:53.677795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf30edd-39fd-4c13-88e6-f60893c605a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:53.678240 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf30edd-39fd-4c13-88e6-f60893c605a5 None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:53.679624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf30edd-39fd-4c13-88e6-f60893c605a5 None admin] RBAC: Authorizing `identity:get_project(project_id=523fc40cf4044e598d36fb05a352d260)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:53.681448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf30edd-39fd-4c13-88e6-f60893c605a5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:53.683671 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 126/254] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:53 2022] GET /identity/v3/projects/523fc40cf4044e598d36fb05a352d260 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:53.684906 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf30edd-39fd-4c13-88e6-f60893c605a5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:53.746352 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135974c-6d73-4e20-8326-e1cb7847e3ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:53.747758 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135974c-6d73-4e20-8326-e1cb7847e3ad None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:53.792081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:53.795151 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf30edd-39fd-4c13-88e6-f60893c605a5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:53.811242 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:53.833575 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc077b-92df-4cca-8570-0f885eeb3664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:53.835131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc077b-92df-4cca-8570-0f885eeb3664 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:53.835664 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc077b-92df-4cca-8570-0f885eeb3664 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:53.836108 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc077b-92df-4cca-8570-0f885eeb3664 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:53.836965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc077b-92df-4cca-8570-0f885eeb3664 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:53.837551 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc077b-92df-4cca-8570-0f885eeb3664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:53.839118 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc077b-92df-4cca-8570-0f885eeb3664 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:53.848089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 127/255] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:12:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:53.849560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496115d-90b3-437a-92f8-9cba403bc96c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:53.850049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496115d-90b3-437a-92f8-9cba403bc96c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:53.850550 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496115d-90b3-437a-92f8-9cba403bc96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:53.850741 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 129/256] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:12:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:12:53.855335 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135974c-6d73-4e20-8326-e1cb7847e3ad None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:53.873978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f496115d-90b3-437a-92f8-9cba403bc96c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:53.875906 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f496115d-90b3-437a-92f8-9cba403bc96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:53.952864 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 128/257] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:53.954067 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496115d-90b3-437a-92f8-9cba403bc96c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:53.954662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135974c-6d73-4e20-8326-e1cb7847e3ad None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:53.999468 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f8ffd-3dda-402b-881b-aafcaea4ebd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:54.000538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f8ffd-3dda-402b-881b-aafcaea4ebd2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:54.001208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f8ffd-3dda-402b-881b-aafcaea4ebd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:54.001575 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f8ffd-3dda-402b-881b-aafcaea4ebd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:54.002426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f8ffd-3dda-402b-881b-aafcaea4ebd2 None admin] RBAC: Adding query filter params (role.id=3b1d23e3e45d44a3a8069b8e823410b5, scope.project.id=523fc40cf4044e598d36fb05a352d260, user.id=6ec6b5f76e9c4f16a5565ea1629b3563) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:54.002983 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f8ffd-3dda-402b-881b-aafcaea4ebd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:54.005089 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f8ffd-3dda-402b-881b-aafcaea4ebd2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:54.012203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 130/258] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:12:53 2022] GET /identity/v3/role_assignments?user.id=6ec6b5f76e9c4f16a5565ea1629b3563&scope.project.id=523fc40cf4044e598d36fb05a352d260&role.id=3b1d23e3e45d44a3a8069b8e823410b5 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:54.015205 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f8ffd-3dda-402b-881b-aafcaea4ebd2 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:54.043104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:54.133908 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496115d-90b3-437a-92f8-9cba403bc96c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:54.142266 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:54.175315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f8ffd-3dda-402b-881b-aafcaea4ebd2 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:54.185955 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496ad67-e3db-436f-a878-ad066fcf39a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:54.186976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496ad67-e3db-436f-a878-ad066fcf39a0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:54.187487 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496ad67-e3db-436f-a878-ad066fcf39a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:54.187954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496ad67-e3db-436f-a878-ad066fcf39a0 None admin] PATH_INFO: `/v3/users/1551c4b972544523bdf81960f5e512d9` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:54.195060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496ad67-e3db-436f-a878-ad066fcf39a0 None admin] RBAC: Authorizing `identity:get_user(user_id=1551c4b972544523bdf81960f5e512d9)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:54.196964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496ad67-e3db-436f-a878-ad066fcf39a0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:54.199264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 129/259] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:53 2022] GET /identity/v3/users/1551c4b972544523bdf81960f5e512d9 => generated 297 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:54.200332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496ad67-e3db-436f-a878-ad066fcf39a0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:54.214293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182bb59-e23c-47c4-839f-64891fe92932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:54.214293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182bb59-e23c-47c4-839f-64891fe92932 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:54.214293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182bb59-e23c-47c4-839f-64891fe92932 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:54.214293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182bb59-e23c-47c4-839f-64891fe92932 None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:54.214694 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182bb59-e23c-47c4-839f-64891fe92932 None admin] RBAC: Authorizing `identity:get_project(project_id=523fc40cf4044e598d36fb05a352d260)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:54.216261 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182bb59-e23c-47c4-839f-64891fe92932 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:54.217441 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 131/260] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:54 2022] GET /identity/v3/projects/523fc40cf4044e598d36fb05a352d260 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:54.218220 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182bb59-e23c-47c4-839f-64891fe92932 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:54.314751 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:54.347777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496ad67-e3db-436f-a878-ad066fcf39a0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:54.386459 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090efcae-1e0f-4a03-bef0-68d7497c2499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:54.387714 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090efcae-1e0f-4a03-bef0-68d7497c2499 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:54.388194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090efcae-1e0f-4a03-bef0-68d7497c2499 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:54.388790 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090efcae-1e0f-4a03-bef0-68d7497c2499 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:54.398149 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182bb59-e23c-47c4-839f-64891fe92932 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:54.411836 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090efcae-1e0f-4a03-bef0-68d7497c2499 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:54.416737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-090efcae-1e0f-4a03-bef0-68d7497c2499 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:12:54.416737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:54.416737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:54.416737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:54.416737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:54.416737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:54.416737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:54.416737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:54.416737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:54.416737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:54.416737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:54.416737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:54.416737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:54.416737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:54.416737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:54.416737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:54.417613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:54.417613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:54.417613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:54.417613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:54.417613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:54.417613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:54.417613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:54.417613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:54.417613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:54.417613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:54.417613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:54.417613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:54.417613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:54.417613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:54.417613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:54.417613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:54.418478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:54.418478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:54.418478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:54.418478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:54.418478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:54.418478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:54.418478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:54.418478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:54.418478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:54.418478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:54.418478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:12:54.418478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:54.424481 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 130/261] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:12:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:54.425324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090efcae-1e0f-4a03-bef0-68d7497c2499 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:54.453045 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e26e0e-6d42-464e-aece-6ee1f4d01fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:54.453935 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e26e0e-6d42-464e-aece-6ee1f4d01fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:54.454493 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e26e0e-6d42-464e-aece-6ee1f4d01fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:54.454990 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e26e0e-6d42-464e-aece-6ee1f4d01fa1 None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:54.456902 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e26e0e-6d42-464e-aece-6ee1f4d01fa1 None admin] RBAC: Authorizing `identity:get_project(project_id=523fc40cf4044e598d36fb05a352d260)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:54.458587 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e26e0e-6d42-464e-aece-6ee1f4d01fa1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:54.461092 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 132/262] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:54 2022] GET /identity/v3/projects/523fc40cf4044e598d36fb05a352d260 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:54.462322 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e26e0e-6d42-464e-aece-6ee1f4d01fa1 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:54.521124 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:54.578167 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e26e0e-6d42-464e-aece-6ee1f4d01fa1 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:54.616318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6459c-e59b-4397-ae04-886345c7b96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:54.617272 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6459c-e59b-4397-ae04-886345c7b96d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:54.617765 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6459c-e59b-4397-ae04-886345c7b96d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:54.618278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6459c-e59b-4397-ae04-886345c7b96d None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:54.619306 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6459c-e59b-4397-ae04-886345c7b96d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:54.619923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6459c-e59b-4397-ae04-886345c7b96d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:54.621495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6459c-e59b-4397-ae04-886345c7b96d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:54.637385 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 133/263] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:12:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:54.638620 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6459c-e59b-4397-ae04-886345c7b96d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:54.670629 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:54.732688 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090efcae-1e0f-4a03-bef0-68d7497c2499 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:54.775445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6459c-e59b-4397-ae04-886345c7b96d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:54.803529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375aac4-7c22-45a1-93c6-559c5607d8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:54.804183 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375aac4-7c22-45a1-93c6-559c5607d8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:54.804467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375aac4-7c22-45a1-93c6-559c5607d8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:54.804736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375aac4-7c22-45a1-93c6-559c5607d8e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:54.805465 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375aac4-7c22-45a1-93c6-559c5607d8e5 None admin] RBAC: Adding query filter params (role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.project.id=523fc40cf4044e598d36fb05a352d260, user.id=90b72abbdbb34f7d9d5aa427e3749154) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:54.805775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375aac4-7c22-45a1-93c6-559c5607d8e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:54.808181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375aac4-7c22-45a1-93c6-559c5607d8e5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:54.823031 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 131/264] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:12:54 2022] GET /identity/v3/role_assignments?user.id=90b72abbdbb34f7d9d5aa427e3749154&scope.project.id=523fc40cf4044e598d36fb05a352d260&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 608 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:54.823876 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375aac4-7c22-45a1-93c6-559c5607d8e5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:54.849096 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94cd87-ead8-41b3-9c03-2132e993b4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:54.850235 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94cd87-ead8-41b3-9c03-2132e993b4f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:54.850549 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94cd87-ead8-41b3-9c03-2132e993b4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:54.850825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94cd87-ead8-41b3-9c03-2132e993b4f0 None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260/users/1551c4b972544523bdf81960f5e512d9/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:54.856095 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94cd87-ead8-41b3-9c03-2132e993b4f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=523fc40cf4044e598d36fb05a352d260, user_id=1551c4b972544523bdf81960f5e512d9, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:54.857594 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94cd87-ead8-41b3-9c03-2132e993b4f0 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:54.886563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 134/265] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:12:54 2022] PUT /identity/v3/projects/523fc40cf4044e598d36fb05a352d260/users/1551c4b972544523bdf81960f5e512d9/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:12:54.967868 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:54.972243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375aac4-7c22-45a1-93c6-559c5607d8e5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:55.011469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a641f-bef4-477d-9ba6-f777a9a24a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:55.012112 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a641f-bef4-477d-9ba6-f777a9a24a02 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:55.012392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a641f-bef4-477d-9ba6-f777a9a24a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:55.012667 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a641f-bef4-477d-9ba6-f777a9a24a02 None admin] PATH_INFO: `/v3/users/90b72abbdbb34f7d9d5aa427e3749154` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:55.013989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a641f-bef4-477d-9ba6-f777a9a24a02 None admin] RBAC: Authorizing `identity:get_user(user_id=90b72abbdbb34f7d9d5aa427e3749154)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:55.015600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a641f-bef4-477d-9ba6-f777a9a24a02 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:55.022505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 132/266] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:54 2022] GET /identity/v3/users/90b72abbdbb34f7d9d5aa427e3749154 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:55.024209 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94cd87-ead8-41b3-9c03-2132e993b4f0 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:55.053080 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5013a40-f5fc-4348-a11b-061437b6b6a1 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:55.053388 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5013a40-f5fc-4348-a11b-061437b6b6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:55.053631 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5013a40-f5fc-4348-a11b-061437b6b6a1 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:55.055212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 133/267] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:12:55.060735 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af9f8b-c869-488d-b454-0afba5968e63 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:55.061024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af9f8b-c869-488d-b454-0afba5968e63 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:55.061344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af9f8b-c869-488d-b454-0afba5968e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:55.084176 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-29af9f8b-c869-488d-b454-0afba5968e63 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:55.084176 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-29af9f8b-c869-488d-b454-0afba5968e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:55.130995 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:55.168063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 134/268] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:12:55.169542 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143789b8-1f28-4b36-93c8-31f3347569e2 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:55.169827 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143789b8-1f28-4b36-93c8-31f3347569e2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:55.170099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143789b8-1f28-4b36-93c8-31f3347569e2 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:55.176067 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 135/269] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:55.181839 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78fd77-f92c-48c3-974a-a4fff75fda32 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:55.182125 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78fd77-f92c-48c3-974a-a4fff75fda32 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:55.182445 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78fd77-f92c-48c3-974a-a4fff75fda32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:55.197939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94cd87-ead8-41b3-9c03-2132e993b4f0 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:55.210844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5a78fd77-f92c-48c3-974a-a4fff75fda32 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:55.212656 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5a78fd77-f92c-48c3-974a-a4fff75fda32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:55.240162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b3c2b-c49a-4af7-ba4e-895ee6df034b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:55.240850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b3c2b-c49a-4af7-ba4e-895ee6df034b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:55.241135 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b3c2b-c49a-4af7-ba4e-895ee6df034b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:55.241414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b3c2b-c49a-4af7-ba4e-895ee6df034b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:55.302030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 136/270] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:12:55.302940 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78fd77-f92c-48c3-974a-a4fff75fda32 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:55.333657 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:55.341811 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b3c2b-c49a-4af7-ba4e-895ee6df034b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:55.343343 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b3c2b-c49a-4af7-ba4e-895ee6df034b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:55.422924 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78fd77-f92c-48c3-974a-a4fff75fda32 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:55.423547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:55.466535 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a404eb-89e0-452a-9b9e-3b87e11c8a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:55.467177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a404eb-89e0-452a-9b9e-3b87e11c8a2f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:55.467457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a404eb-89e0-452a-9b9e-3b87e11c8a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:55.467729 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a404eb-89e0-452a-9b9e-3b87e11c8a2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:55.469343 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a404eb-89e0-452a-9b9e-3b87e11c8a2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:55.471097 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a404eb-89e0-452a-9b9e-3b87e11c8a2f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:55.478279 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 137/271] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:12:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:55.478279 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a404eb-89e0-452a-9b9e-3b87e11c8a2f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:55.478484 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 135/272] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:12:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:12:55.478484 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b3c2b-c49a-4af7-ba4e-895ee6df034b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:55.607054 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b3c2b-c49a-4af7-ba4e-895ee6df034b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:55.608155 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:55.619693 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a404eb-89e0-452a-9b9e-3b87e11c8a2f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:55.645985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d3d41-b94b-4e32-9c47-f7a7dc3527cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:55.647196 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d3d41-b94b-4e32-9c47-f7a7dc3527cc None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:55.647472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d3d41-b94b-4e32-9c47-f7a7dc3527cc None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:55.647740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d3d41-b94b-4e32-9c47-f7a7dc3527cc None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:55.648925 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d3d41-b94b-4e32-9c47-f7a7dc3527cc None admin] RBAC: Authorizing `identity:get_project(project_id=523fc40cf4044e598d36fb05a352d260)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:55.650449 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d3d41-b94b-4e32-9c47-f7a7dc3527cc None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:55.652430 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 136/273] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:55 2022] GET /identity/v3/projects/523fc40cf4044e598d36fb05a352d260 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:55.653251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d3d41-b94b-4e32-9c47-f7a7dc3527cc None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:55.672503 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fb2e7-ef9b-4590-9127-276d629723de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:55.673507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fb2e7-ef9b-4590-9127-276d629723de None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:55.673736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fb2e7-ef9b-4590-9127-276d629723de None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:55.674013 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fb2e7-ef9b-4590-9127-276d629723de None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:55.689227 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fb2e7-ef9b-4590-9127-276d629723de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:55.696282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:55.697336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7d1fb2e7-ef9b-4590-9127-276d629723de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:55.697336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:55.697336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:55.697336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:55.697336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:55.697336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:55.697336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:55.697336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:55.697336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:55.697336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:55.697336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:55.697336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:55.697336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:55.697336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:55.697336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:55.697336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:55.699450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:55.699450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:55.699450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:55.699450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:55.699450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:55.699450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:55.699450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:55.699450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:55.699450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:55.699450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:55.699450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:55.699450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:55.699450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:55.699450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:55.699450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:55.699450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:55.700509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:55.700509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:55.700509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:55.700509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:55.700509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:55.700509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:55.700509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:55.700509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:55.700509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:55.700509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:55.700509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:55.700509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:55.706960 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 138/274] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:55.707307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fb2e7-ef9b-4590-9127-276d629723de None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:55.791505 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d3d41-b94b-4e32-9c47-f7a7dc3527cc None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:55.823421 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fb2e7-ef9b-4590-9127-276d629723de None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:55.838195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e825b-b35b-46a3-9881-62fddb127031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:55.838702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e825b-b35b-46a3-9881-62fddb127031 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:55.838975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e825b-b35b-46a3-9881-62fddb127031 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:55.839242 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e825b-b35b-46a3-9881-62fddb127031 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:55.839962 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e825b-b35b-46a3-9881-62fddb127031 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:55.841361 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e825b-b35b-46a3-9881-62fddb127031 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:55.867385 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975d74a-625e-418e-a5f8-c0c8182aa79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:55.868363 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975d74a-625e-418e-a5f8-c0c8182aa79a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:55.868865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975d74a-625e-418e-a5f8-c0c8182aa79a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:55.869337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975d74a-625e-418e-a5f8-c0c8182aa79a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:55.870326 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975d74a-625e-418e-a5f8-c0c8182aa79a None admin] RBAC: Adding query filter params (role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839, scope.project.id=523fc40cf4044e598d36fb05a352d260, user.id=90b72abbdbb34f7d9d5aa427e3749154) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:55.870450 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 137/275] 10.222.0.39 () {60 vars in 1132 bytes} [Mon Aug 8 02:12:55 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:12:55.870986 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975d74a-625e-418e-a5f8-c0c8182aa79a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:55.871339 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e825b-b35b-46a3-9881-62fddb127031 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:55.872917 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975d74a-625e-418e-a5f8-c0c8182aa79a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:55.880130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 139/276] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:12:55 2022] GET /identity/v3/role_assignments?user.id=90b72abbdbb34f7d9d5aa427e3749154&scope.project.id=523fc40cf4044e598d36fb05a352d260&role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:55.999065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:56.010416 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e825b-b35b-46a3-9881-62fddb127031 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:56.055080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d3c30-bac3-4fb5-af75-929c10d70b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:56.055855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d3c30-bac3-4fb5-af75-929c10d70b1d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:56.056113 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d3c30-bac3-4fb5-af75-929c10d70b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:56.056384 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d3c30-bac3-4fb5-af75-929c10d70b1d None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:56.057077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d3c30-bac3-4fb5-af75-929c10d70b1d None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:56.057380 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d3c30-bac3-4fb5-af75-929c10d70b1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:56.058824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d3c30-bac3-4fb5-af75-929c10d70b1d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:56.077165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 138/277] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:12:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:56.087466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975d74a-625e-418e-a5f8-c0c8182aa79a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:56.122064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:56.212353 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975d74a-625e-418e-a5f8-c0c8182aa79a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:56.246850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e47804-2794-4d2a-b450-144cb7a90284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:56.247481 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e47804-2794-4d2a-b450-144cb7a90284 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:56.247824 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e47804-2794-4d2a-b450-144cb7a90284 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:56.248157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e47804-2794-4d2a-b450-144cb7a90284 None admin] PATH_INFO: `/v3/users/76cb3c6199b140419ac3d66cfb278e04` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:56.264709 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e47804-2794-4d2a-b450-144cb7a90284 None admin] RBAC: Authorizing `identity:get_user(user_id=76cb3c6199b140419ac3d66cfb278e04)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:56.266189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e47804-2794-4d2a-b450-144cb7a90284 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:56.268307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 140/278] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:56 2022] GET /identity/v3/users/76cb3c6199b140419ac3d66cfb278e04 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:56.274898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d3c30-bac3-4fb5-af75-929c10d70b1d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:56.388779 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d3c30-bac3-4fb5-af75-929c10d70b1d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:56.425988 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170c143-73c9-41be-9e26-eec6917f17e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:56.426618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170c143-73c9-41be-9e26-eec6917f17e7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:56.426898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170c143-73c9-41be-9e26-eec6917f17e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:56.427226 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170c143-73c9-41be-9e26-eec6917f17e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:56.428535 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170c143-73c9-41be-9e26-eec6917f17e7 None admin] RBAC: Adding query filter params (role.id=42c846f6944b40c8af5eeb8360cb7c7a, user.id=76cb3c6199b140419ac3d66cfb278e04, scope.system=all) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:56.428845 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170c143-73c9-41be-9e26-eec6917f17e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:56.430381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170c143-73c9-41be-9e26-eec6917f17e7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:56.435945 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 139/279] 10.222.0.39 () {56 vars in 1308 bytes} [Mon Aug 8 02:12:56 2022] GET /identity/v3/role_assignments?user.id=76cb3c6199b140419ac3d66cfb278e04&scope.system=all&role.id=42c846f6944b40c8af5eeb8360cb7c7a => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:56.623667 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad99b05-1b29-4f4e-899d-d68db0eff120 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:56.624080 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad99b05-1b29-4f4e-899d-d68db0eff120 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:56.624457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad99b05-1b29-4f4e-899d-d68db0eff120 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:56.626167 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 141/280] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:56.632009 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e266745-7383-4137-aecb-831b2fb0f2f0 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:56.632478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e266745-7383-4137-aecb-831b2fb0f2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:56.632924 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e266745-7383-4137-aecb-831b2fb0f2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:56.654366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-4e266745-7383-4137-aecb-831b2fb0f2f0 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:56.655795 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4e266745-7383-4137-aecb-831b2fb0f2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:56.668559 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:56.751607 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 140/281] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:56.773909 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad99b05-1b29-4f4e-899d-d68db0eff120 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:56.884345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:56.914909 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad99b05-1b29-4f4e-899d-d68db0eff120 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:56.960693 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd27ff-cd42-4bda-9bad-65a466f95b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:56.961673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd27ff-cd42-4bda-9bad-65a466f95b84 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:56.962143 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd27ff-cd42-4bda-9bad-65a466f95b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:56.962649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd27ff-cd42-4bda-9bad-65a466f95b84 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:56.976583 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd27ff-cd42-4bda-9bad-65a466f95b84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:56.981514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-21cd27ff-cd42-4bda-9bad-65a466f95b84 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:12:56.981514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:56.981514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:56.981514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:56.981514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:56.981514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:56.981514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:56.981514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:56.981514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:56.981514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:56.981514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:56.981514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:56.981514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:56.981514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:56.981514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:56.981514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:56.986238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:56.986238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:56.986238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:56.986238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:56.986238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:56.986238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:56.986238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:56.986238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:56.986238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:56.986238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:56.986238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:56.986238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:56.986238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:56.986238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:56.986238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:56.986238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:56.987048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:56.987048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:56.987048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:56.987048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:56.987048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:56.987048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:56.987048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:56.987048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:56.987048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:56.987048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:56.987048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:12:56.987048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:56.989471 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 142/282] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:56.994968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e266745-7383-4137-aecb-831b2fb0f2f0 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:57.126572 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e266745-7383-4137-aecb-831b2fb0f2f0 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:57.171276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ad192-3c90-4648-a3ff-121d2327d14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:57.171809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ad192-3c90-4648-a3ff-121d2327d14e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:57.172090 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ad192-3c90-4648-a3ff-121d2327d14e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:57.172366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ad192-3c90-4648-a3ff-121d2327d14e None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:57.173098 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ad192-3c90-4648-a3ff-121d2327d14e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:57.173404 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ad192-3c90-4648-a3ff-121d2327d14e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:57.174880 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ad192-3c90-4648-a3ff-121d2327d14e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:57.188520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 141/283] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:12:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:57.194731 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd27ff-cd42-4bda-9bad-65a466f95b84 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:57.283140 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971055c-c138-4f4d-8f48-e12b5d94b809 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:57.283459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971055c-c138-4f4d-8f48-e12b5d94b809 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:57.283702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971055c-c138-4f4d-8f48-e12b5d94b809 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:57.285298 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 142/284] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:12:57.290260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef139152-4af2-4ddf-b1f0-32a3da43887b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:57.290571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef139152-4af2-4ddf-b1f0-32a3da43887b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:57.290844 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef139152-4af2-4ddf-b1f0-32a3da43887b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:57.315030 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ef139152-4af2-4ddf-b1f0-32a3da43887b None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:57.316231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ef139152-4af2-4ddf-b1f0-32a3da43887b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:57.321486 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:57.367144 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd27ff-cd42-4bda-9bad-65a466f95b84 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:57.431713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b968ba8-04c5-43d8-99d4-448a0fff680d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:57.432264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b968ba8-04c5-43d8-99d4-448a0fff680d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:57.432543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b968ba8-04c5-43d8-99d4-448a0fff680d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:57.432812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b968ba8-04c5-43d8-99d4-448a0fff680d None admin] PATH_INFO: `/v3/users/6ec6b5f76e9c4f16a5565ea1629b3563` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:57.435368 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b968ba8-04c5-43d8-99d4-448a0fff680d None admin] RBAC: Authorizing `identity:get_user(user_id=6ec6b5f76e9c4f16a5565ea1629b3563)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:57.436831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b968ba8-04c5-43d8-99d4-448a0fff680d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:57.440049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 143/285] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:57 2022] GET /identity/v3/users/6ec6b5f76e9c4f16a5565ea1629b3563 => generated 297 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:57.449261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b968ba8-04c5-43d8-99d4-448a0fff680d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:57.450723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 143/286] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:12:57.459327 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:57.475151 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef139152-4af2-4ddf-b1f0-32a3da43887b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:57.585602 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef139152-4af2-4ddf-b1f0-32a3da43887b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:57.639917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6a928-dbfb-430f-9024-5fb298bd462b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:57.640546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6a928-dbfb-430f-9024-5fb298bd462b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:57.640821 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6a928-dbfb-430f-9024-5fb298bd462b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:57.641087 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6a928-dbfb-430f-9024-5fb298bd462b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:57.659408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:57.664643 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6a928-dbfb-430f-9024-5fb298bd462b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:57.669399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-2dc6a928-dbfb-430f-9024-5fb298bd462b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:57.669399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:57.669399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:57.669399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:57.669399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:57.669399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:57.669399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:57.669399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:57.669399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:57.669399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:57.669399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:57.669399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:57.669399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:57.669399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:57.669399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:57.669399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:57.670345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:57.670345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:57.670345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:57.670345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:57.670345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:57.670345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:57.670345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:57.670345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:57.670345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:57.670345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:57.670345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:57.670345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:57.670345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:57.670345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:57.670345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:57.670345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:57.671114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:57.671114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:57.671114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:57.671114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:57.671114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:57.671114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:57.671114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:57.671114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:57.671114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:57.671114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:57.671114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:57.671114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:12:57.673998 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 144/287] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:57.679613 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6a928-dbfb-430f-9024-5fb298bd462b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:57.692801 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b968ba8-04c5-43d8-99d4-448a0fff680d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:57.757460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea6a3d-c879-44e5-a529-eb66fec56d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:57.758533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea6a3d-c879-44e5-a529-eb66fec56d04 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:57.759018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea6a3d-c879-44e5-a529-eb66fec56d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:57.759603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea6a3d-c879-44e5-a529-eb66fec56d04 None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:57.763698 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea6a3d-c879-44e5-a529-eb66fec56d04 None admin] RBAC: Authorizing `identity:get_project(project_id=523fc40cf4044e598d36fb05a352d260)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:57.765469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea6a3d-c879-44e5-a529-eb66fec56d04 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:57.767848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 144/288] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:57 2022] GET /identity/v3/projects/523fc40cf4044e598d36fb05a352d260 => generated 302 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:57.773892 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea6a3d-c879-44e5-a529-eb66fec56d04 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:57.854599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6a928-dbfb-430f-9024-5fb298bd462b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:57.860157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:57.896854 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea6a3d-c879-44e5-a529-eb66fec56d04 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:57.938457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed3b3e-0dfe-4d46-bcca-697657f50d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:57.939357 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df02e7-818f-4488-be6a-5295a538e793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:57.940204 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed3b3e-0dfe-4d46-bcca-697657f50d01 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:57.940974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df02e7-818f-4488-be6a-5295a538e793 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:57.941452 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed3b3e-0dfe-4d46-bcca-697657f50d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:57.941901 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df02e7-818f-4488-be6a-5295a538e793 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:57.942368 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed3b3e-0dfe-4d46-bcca-697657f50d01 None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260/users/6ec6b5f76e9c4f16a5565ea1629b3563/roles/3b1d23e3e45d44a3a8069b8e823410b5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:57.942868 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df02e7-818f-4488-be6a-5295a538e793 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:57.944517 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df02e7-818f-4488-be6a-5295a538e793 None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:57.945230 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df02e7-818f-4488-be6a-5295a538e793 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:57.947136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df02e7-818f-4488-be6a-5295a538e793 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:57.947970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed3b3e-0dfe-4d46-bcca-697657f50d01 None admin] RBAC: Authorizing `identity:create_grant(project_id=523fc40cf4044e598d36fb05a352d260, user_id=6ec6b5f76e9c4f16a5565ea1629b3563, role_id=3b1d23e3e45d44a3a8069b8e823410b5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:57.949569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed3b3e-0dfe-4d46-bcca-697657f50d01 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:57.966597 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 145/289] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:12:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:57.969204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df02e7-818f-4488-be6a-5295a538e793 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:57.983544 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 145/290] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:12:57 2022] PUT /identity/v3/projects/523fc40cf4044e598d36fb05a352d260/users/6ec6b5f76e9c4f16a5565ea1629b3563/roles/3b1d23e3e45d44a3a8069b8e823410b5 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:12:58.044575 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:58.092364 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df02e7-818f-4488-be6a-5295a538e793 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:58.130789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f232b-7a79-4f63-9e01-0adfce651aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:58.132191 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f232b-7a79-4f63-9e01-0adfce651aec None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:58.132684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f232b-7a79-4f63-9e01-0adfce651aec None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:58.133137 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f232b-7a79-4f63-9e01-0adfce651aec None admin] PATH_INFO: `/v3/users/1551c4b972544523bdf81960f5e512d9` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:58.134587 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f232b-7a79-4f63-9e01-0adfce651aec None admin] RBAC: Authorizing `identity:get_user(user_id=1551c4b972544523bdf81960f5e512d9)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:58.136223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f232b-7a79-4f63-9e01-0adfce651aec None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:58.138374 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 146/291] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:57 2022] GET /identity/v3/users/1551c4b972544523bdf81960f5e512d9 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:58.144328 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed3b3e-0dfe-4d46-bcca-697657f50d01 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:58.255847 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed3b3e-0dfe-4d46-bcca-697657f50d01 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:58.291145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:58.298831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9836403-8842-495d-a5a5-4abfabe387b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:58.299682 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9836403-8842-495d-a5a5-4abfabe387b1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:58.300243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9836403-8842-495d-a5a5-4abfabe387b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:58.300692 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9836403-8842-495d-a5a5-4abfabe387b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:58.393042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9836403-8842-495d-a5a5-4abfabe387b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:58.394823 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9836403-8842-495d-a5a5-4abfabe387b1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:58.543130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 146/292] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:12:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:12:58.548885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f232b-7a79-4f63-9e01-0adfce651aec None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:58.688495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f232b-7a79-4f63-9e01-0adfce651aec None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:58.716502 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:58.732368 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d9f84-fb7d-4427-91fc-7babf693d768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:58.733008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d9f84-fb7d-4427-91fc-7babf693d768 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:58.733289 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d9f84-fb7d-4427-91fc-7babf693d768 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:58.733565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d9f84-fb7d-4427-91fc-7babf693d768 None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:58.734816 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d9f84-fb7d-4427-91fc-7babf693d768 None admin] RBAC: Authorizing `identity:get_project(project_id=523fc40cf4044e598d36fb05a352d260)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:58.736264 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d9f84-fb7d-4427-91fc-7babf693d768 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:58.737916 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 147/293] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:12:58 2022] GET /identity/v3/projects/523fc40cf4044e598d36fb05a352d260 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:58.744541 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9836403-8842-495d-a5a5-4abfabe387b1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:58.777643 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3b90c-d7fa-4398-bdf6-42c9c71f6101 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:58.777922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3b90c-d7fa-4398-bdf6-42c9c71f6101 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:58.778200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3b90c-d7fa-4398-bdf6-42c9c71f6101 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:58.779630 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 148/294] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:12:58.784953 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87326f29-2192-4d7e-809d-65c4553450cb None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:58.785237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87326f29-2192-4d7e-809d-65c4553450cb None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:58.785511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87326f29-2192-4d7e-809d-65c4553450cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:58.803303 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:58.815262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-87326f29-2192-4d7e-809d-65c4553450cb None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:58.816478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-87326f29-2192-4d7e-809d-65c4553450cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:58.880148 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9836403-8842-495d-a5a5-4abfabe387b1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:58.924384 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85d713-5272-4a75-8094-24c6cb205ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:58.924989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85d713-5272-4a75-8094-24c6cb205ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:58.925267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85d713-5272-4a75-8094-24c6cb205ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:58.925538 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85d713-5272-4a75-8094-24c6cb205ab5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:58.926274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85d713-5272-4a75-8094-24c6cb205ab5 None admin] RBAC: Adding query filter params (role.id=42c846f6944b40c8af5eeb8360cb7c7a, scope.project.id=523fc40cf4044e598d36fb05a352d260, user.id=1551c4b972544523bdf81960f5e512d9) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:58.926598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85d713-5272-4a75-8094-24c6cb205ab5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:58.927939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 149/295] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:12:58.928576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85d713-5272-4a75-8094-24c6cb205ab5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:58.929185 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d619e5-387e-472d-ad2f-576328232085 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:58.929465 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d619e5-387e-472d-ad2f-576328232085 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:58.929732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d619e5-387e-472d-ad2f-576328232085 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:58.931257 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 150/296] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:58.936837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8feda81-c4ee-4520-a5c1-ac63f62cef02 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:58.937120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8feda81-c4ee-4520-a5c1-ac63f62cef02 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:58.937393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8feda81-c4ee-4520-a5c1-ac63f62cef02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:58.944422 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 147/297] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:12:58 2022] GET /identity/v3/role_assignments?user.id=1551c4b972544523bdf81960f5e512d9&scope.project.id=523fc40cf4044e598d36fb05a352d260&role.id=42c846f6944b40c8af5eeb8360cb7c7a => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:58.963636 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85d713-5272-4a75-8094-24c6cb205ab5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:58.974288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a8feda81-c4ee-4520-a5c1-ac63f62cef02 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:58.975586 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a8feda81-c4ee-4520-a5c1-ac63f62cef02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:59.064985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 151/298] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:12:59.066559 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3347fd2-eca1-4ee0-81c0-b2d6a1da5ac4 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:59.067039 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3347fd2-eca1-4ee0-81c0-b2d6a1da5ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:59.067358 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3347fd2-eca1-4ee0-81c0-b2d6a1da5ac4 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:59.069266 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 152/299] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:12:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:12:59.075608 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4e1bb-6a8a-45dd-9c00-d646e9f072f8 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:59.075977 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4e1bb-6a8a-45dd-9c00-d646e9f072f8 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:59.076324 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4e1bb-6a8a-45dd-9c00-d646e9f072f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:59.096194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-5be4e1bb-6a8a-45dd-9c00-d646e9f072f8 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:12:59.096561 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:59.098769 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5be4e1bb-6a8a-45dd-9c00-d646e9f072f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:12:59.104612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85d713-5272-4a75-8094-24c6cb205ab5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:59.147966 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d6a6b-a22f-415d-83bb-d48ed7d4acd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:59.148595 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d6a6b-a22f-415d-83bb-d48ed7d4acd5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:59.148869 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d6a6b-a22f-415d-83bb-d48ed7d4acd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:59.149133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d6a6b-a22f-415d-83bb-d48ed7d4acd5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:59.163066 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d6a6b-a22f-415d-83bb-d48ed7d4acd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:59.163230 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:59.168707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7c0d6a6b-a22f-415d-83bb-d48ed7d4acd5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:12:59.168707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:59.168707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:59.168707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:59.168707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:59.168707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:59.168707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:59.168707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:59.168707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:59.168707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:59.168707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:59.168707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:59.168707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:59.168707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:59.168707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:59.168707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:59.169563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:59.169563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:59.169563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:59.169563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:59.169563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:59.169563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:59.169563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:59.169563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:59.169563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:59.169563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:59.169563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:59.169563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:59.169563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:59.169563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:59.169563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:59.169563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:59.170684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:59.170684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:59.170684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:59.170684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:59.170684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:59.170684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:59.170684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:59.170684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:59.170684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:59.170684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:59.170684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:12:59.170684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:59.170684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 148/300] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:59.171483 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d6a6b-a22f-415d-83bb-d48ed7d4acd5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:59.186534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 153/301] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:12:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:12:59.187111 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4e1bb-6a8a-45dd-9c00-d646e9f072f8 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:59.292003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d6a6b-a22f-415d-83bb-d48ed7d4acd5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:59.307790 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4e1bb-6a8a-45dd-9c00-d646e9f072f8 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:59.326416 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e21936-e4ea-41dd-8d57-96f7eea7e57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:59.326974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e21936-e4ea-41dd-8d57-96f7eea7e57c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:59.327254 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e21936-e4ea-41dd-8d57-96f7eea7e57c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:59.327527 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e21936-e4ea-41dd-8d57-96f7eea7e57c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:59.328627 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e21936-e4ea-41dd-8d57-96f7eea7e57c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:59.330061 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e21936-e4ea-41dd-8d57-96f7eea7e57c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:59.336145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 149/302] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:12:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:59.336145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e21936-e4ea-41dd-8d57-96f7eea7e57c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:59.350551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae006d3-75e3-4a1e-99c6-14cf282d1d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:59.351402 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae006d3-75e3-4a1e-99c6-14cf282d1d23 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:59.351819 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae006d3-75e3-4a1e-99c6-14cf282d1d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:59.352234 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae006d3-75e3-4a1e-99c6-14cf282d1d23 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:59.352891 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae006d3-75e3-4a1e-99c6-14cf282d1d23 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:59.353190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae006d3-75e3-4a1e-99c6-14cf282d1d23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:59.354575 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae006d3-75e3-4a1e-99c6-14cf282d1d23 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:59.358433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:59.368000 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 154/303] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:12:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:59.368000 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae006d3-75e3-4a1e-99c6-14cf282d1d23 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:59.415872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:59.446549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e21936-e4ea-41dd-8d57-96f7eea7e57c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:59.479747 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae006d3-75e3-4a1e-99c6-14cf282d1d23 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:59.483937 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6967fdc-0871-48bd-b733-2385b74fce16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:59.484564 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6967fdc-0871-48bd-b733-2385b74fce16 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:59.484839 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6967fdc-0871-48bd-b733-2385b74fce16 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:59.485106 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6967fdc-0871-48bd-b733-2385b74fce16 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:59.495577 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6967fdc-0871-48bd-b733-2385b74fce16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:12:59.500259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e6967fdc-0871-48bd-b733-2385b74fce16 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:59.500259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:12:59.500259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:12:59.500259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:12:59.500259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:12:59.500259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:12:59.500259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:12:59.500259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:12:59.500259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:12:59.500259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:12:59.500259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:12:59.500259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:12:59.500259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:12:59.500259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:12:59.500259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:12:59.500259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:12:59.501110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:12:59.501110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:12:59.501110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:12:59.501110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:12:59.501110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:12:59.501110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:12:59.501110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:12:59.501110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:12:59.501110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:12:59.501110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:12:59.501110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:12:59.501110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:12:59.501110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:12:59.501110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:12:59.501110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:12:59.501110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:12:59.501922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:12:59.501922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:12:59.501922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:12:59.501922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:12:59.501922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:12:59.501922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:12:59.501922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:12:59.501922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:12:59.501922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:12:59.501922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:12:59.501922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:12:59.501922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:12:59.501922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 150/304] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:12:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:12:59.506810 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6967fdc-0871-48bd-b733-2385b74fce16 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:59.516370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6ecc5-63e5-4d42-8163-8bdecafc421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:59.516977 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6ecc5-63e5-4d42-8163-8bdecafc421d None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:59.517587 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6ecc5-63e5-4d42-8163-8bdecafc421d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:59.517887 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6ecc5-63e5-4d42-8163-8bdecafc421d None admin] PATH_INFO: `/v3/groups` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:59.518548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6ecc5-63e5-4d42-8163-8bdecafc421d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:59.520294 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6ecc5-63e5-4d42-8163-8bdecafc421d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:59.537398 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 155/305] 10.222.0.39 () {60 vars in 1134 bytes} [Mon Aug 8 02:12:59 2022] POST /identity/v3/groups => generated 229 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:12:59.547459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6ecc5-63e5-4d42-8163-8bdecafc421d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:59.605160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:59.635011 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6967fdc-0871-48bd-b733-2385b74fce16 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:59.688442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10068d-ec56-4c29-8db8-f69bb7a20344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:59.688887 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6ecc5-63e5-4d42-8163-8bdecafc421d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:59.689064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10068d-ec56-4c29-8db8-f69bb7a20344 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:59.689341 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10068d-ec56-4c29-8db8-f69bb7a20344 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:59.689609 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10068d-ec56-4c29-8db8-f69bb7a20344 None admin] PATH_INFO: `/v3/users/4452f8b7b7ea4c71b682fb0744e92858` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:59.701620 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:12:59.711695 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10068d-ec56-4c29-8db8-f69bb7a20344 None admin] RBAC: Authorizing `identity:get_user(user_id=4452f8b7b7ea4c71b682fb0744e92858)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:59.714225 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10068d-ec56-4c29-8db8-f69bb7a20344 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:59.716182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 151/306] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:59 2022] GET /identity/v3/users/4452f8b7b7ea4c71b682fb0744e92858 => generated 293 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:59.722602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10068d-ec56-4c29-8db8-f69bb7a20344 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:59.733495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976a0db-4e17-4a5f-8ace-f51d470b4d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:59.734804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976a0db-4e17-4a5f-8ace-f51d470b4d9f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:59.734909 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976a0db-4e17-4a5f-8ace-f51d470b4d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:59.735102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976a0db-4e17-4a5f-8ace-f51d470b4d9f None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:59.735772 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976a0db-4e17-4a5f-8ace-f51d470b4d9f None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:59.736078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976a0db-4e17-4a5f-8ace-f51d470b4d9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:59.737807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976a0db-4e17-4a5f-8ace-f51d470b4d9f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:59.750558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 156/307] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:12:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:59.758208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976a0db-4e17-4a5f-8ace-f51d470b4d9f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:59.845892 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10068d-ec56-4c29-8db8-f69bb7a20344 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:59.882716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7413c-a001-47a7-9229-0bebf83937b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:59.883238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7413c-a001-47a7-9229-0bebf83937b4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:59.883515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7413c-a001-47a7-9229-0bebf83937b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:59.883786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7413c-a001-47a7-9229-0bebf83937b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:59.884566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7413c-a001-47a7-9229-0bebf83937b4 None admin] RBAC: Adding query filter params (role.id=3b1d23e3e45d44a3a8069b8e823410b5, user.id=4452f8b7b7ea4c71b682fb0744e92858, scope.system=all) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:12:59.884878 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7413c-a001-47a7-9229-0bebf83937b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:59.886397 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7413c-a001-47a7-9229-0bebf83937b4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:59.889531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976a0db-4e17-4a5f-8ace-f51d470b4d9f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:12:59.891866 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 152/308] 10.222.0.39 () {56 vars in 1308 bytes} [Mon Aug 8 02:12:59 2022] GET /identity/v3/role_assignments?user.id=4452f8b7b7ea4c71b682fb0744e92858&scope.system=all&role.id=3b1d23e3e45d44a3a8069b8e823410b5 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:12:59.937011 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1478e-8599-4ebb-94b6-7104fd566b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:12:59.937657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1478e-8599-4ebb-94b6-7104fd566b4a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:12:59.937933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1478e-8599-4ebb-94b6-7104fd566b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:12:59.938206 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1478e-8599-4ebb-94b6-7104fd566b4a None admin] PATH_INFO: `/v3/users/76cb3c6199b140419ac3d66cfb278e04` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:12:59.939429 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1478e-8599-4ebb-94b6-7104fd566b4a None admin] RBAC: Authorizing `identity:get_user(user_id=76cb3c6199b140419ac3d66cfb278e04)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:12:59.940878 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1478e-8599-4ebb-94b6-7104fd566b4a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:12:59.942849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 157/309] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:12:59 2022] GET /identity/v3/users/76cb3c6199b140419ac3d66cfb278e04 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:12:59.948275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7413c-a001-47a7-9229-0bebf83937b4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:12:59.958714 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:00.065994 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7413c-a001-47a7-9229-0bebf83937b4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:00.102069 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e968e-eeeb-430e-b459-3e23201aae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:00.102748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e968e-eeeb-430e-b459-3e23201aae80 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:00.103027 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e968e-eeeb-430e-b459-3e23201aae80 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:00.103300 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e968e-eeeb-430e-b459-3e23201aae80 None admin] PATH_INFO: `/v3/system/users/76cb3c6199b140419ac3d66cfb278e04/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:00.104777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e968e-eeeb-430e-b459-3e23201aae80 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=76cb3c6199b140419ac3d66cfb278e04, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:00.106171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e968e-eeeb-430e-b459-3e23201aae80 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:00.117064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 153/310] 10.222.0.39 () {58 vars in 1334 bytes} [Mon Aug 8 02:12:59 2022] PUT /identity/v3/system/users/76cb3c6199b140419ac3d66cfb278e04/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:00.378596 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba47a5f-afca-4a6f-994a-2cb308009edc None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:00.379118 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba47a5f-afca-4a6f-994a-2cb308009edc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:00.379118 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba47a5f-afca-4a6f-994a-2cb308009edc None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:00.380777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 158/311] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:00.385686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7005e13-4fa0-428f-8595-8c38bede2242 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:00.386078 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7005e13-4fa0-428f-8595-8c38bede2242 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:00.386240 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7005e13-4fa0-428f-8595-8c38bede2242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:00.404988 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a7005e13-4fa0-428f-8595-8c38bede2242 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:00.406189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a7005e13-4fa0-428f-8595-8c38bede2242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:00.477965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:00.482975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 154/312] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:00.506632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba47a5f-afca-4a6f-994a-2cb308009edc None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:00.548949 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:00.612467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba47a5f-afca-4a6f-994a-2cb308009edc None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:00.648378 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f6f85d-4f78-4f9b-a64a-db82caf4b6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:00.648979 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6f85d-4f78-4f9b-a64a-db82caf4b6f7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:00.649265 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6f85d-4f78-4f9b-a64a-db82caf4b6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:00.649539 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6f85d-4f78-4f9b-a64a-db82caf4b6f7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:00.660114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f6f85d-4f78-4f9b-a64a-db82caf4b6f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:00.665040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-62f6f85d-4f78-4f9b-a64a-db82caf4b6f7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:13:00.665040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:00.665040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:00.665040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:00.665040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:00.665040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:00.665040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:00.665040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:00.665040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:00.665040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:00.665040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:00.665040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:00.665040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:00.665040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:00.665040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:00.665040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:00.665872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:00.665872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:00.665872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:00.665872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:00.665872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:00.665872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:00.665872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:00.665872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:00.665872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:00.665872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:00.665872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:00.665872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:00.665872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:00.665872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:00.665872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:00.665872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:00.666681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:00.666681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:00.666681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:00.666681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:00.666681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:00.666681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:00.666681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:00.666681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:00.666681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:00.666681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:00.666681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:13:00.666681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:00.670594 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 159/313] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:00.672648 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7005e13-4fa0-428f-8595-8c38bede2242 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:00.778885 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7005e13-4fa0-428f-8595-8c38bede2242 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:00.815192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5490f35-6512-4e4c-a58c-43848e019272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:00.815796 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5490f35-6512-4e4c-a58c-43848e019272 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:00.816083 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5490f35-6512-4e4c-a58c-43848e019272 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:00.816357 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5490f35-6512-4e4c-a58c-43848e019272 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:00.817021 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5490f35-6512-4e4c-a58c-43848e019272 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:00.817326 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5490f35-6512-4e4c-a58c-43848e019272 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:00.818782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5490f35-6512-4e4c-a58c-43848e019272 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:00.827292 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 155/314] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:00.832733 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f6f85d-4f78-4f9b-a64a-db82caf4b6f7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:00.930708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:00.935075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f6f85d-4f78-4f9b-a64a-db82caf4b6f7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:00.970333 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54d64d-01b0-435c-993a-399926e1c64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:00.970961 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54d64d-01b0-435c-993a-399926e1c64f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:00.971241 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54d64d-01b0-435c-993a-399926e1c64f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:00.971514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54d64d-01b0-435c-993a-399926e1c64f None admin] PATH_INFO: `/v3/users/6ec6b5f76e9c4f16a5565ea1629b3563` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:00.972715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c54d64d-01b0-435c-993a-399926e1c64f None admin] RBAC: Authorizing `identity:get_user(user_id=6ec6b5f76e9c4f16a5565ea1629b3563)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:00.974828 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c54d64d-01b0-435c-993a-399926e1c64f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:00.976415 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 160/315] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:00 2022] GET /identity/v3/users/6ec6b5f76e9c4f16a5565ea1629b3563 => generated 297 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:00.981994 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5490f35-6512-4e4c-a58c-43848e019272 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:01.049585 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:01.092695 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5490f35-6512-4e4c-a58c-43848e019272 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:01.127807 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74ea65-e7b1-44ba-8adf-354034120bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:01.128462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74ea65-e7b1-44ba-8adf-354034120bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:01.128752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74ea65-e7b1-44ba-8adf-354034120bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:01.129026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74ea65-e7b1-44ba-8adf-354034120bf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:01.202221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74ea65-e7b1-44ba-8adf-354034120bf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:01.203643 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74ea65-e7b1-44ba-8adf-354034120bf3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:01.268283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:01.311018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 156/316] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:01.316435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54d64d-01b0-435c-993a-399926e1c64f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:01.418766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54d64d-01b0-435c-993a-399926e1c64f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:01.454217 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ece33-3505-495f-95b1-e5f2629d4a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:01.454830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ece33-3505-495f-95b1-e5f2629d4a84 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:01.455112 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ece33-3505-495f-95b1-e5f2629d4a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:01.455390 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ece33-3505-495f-95b1-e5f2629d4a84 None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:01.456527 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ece33-3505-495f-95b1-e5f2629d4a84 None admin] RBAC: Authorizing `identity:get_project(project_id=523fc40cf4044e598d36fb05a352d260)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:01.457972 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ece33-3505-495f-95b1-e5f2629d4a84 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:01.460554 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 161/317] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:01 2022] GET /identity/v3/projects/523fc40cf4044e598d36fb05a352d260 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:01.466381 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74ea65-e7b1-44ba-8adf-354034120bf3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:01.567566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74ea65-e7b1-44ba-8adf-354034120bf3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:01.593655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792b442-7485-460e-8733-090b0d6576aa None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:01.593970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792b442-7485-460e-8733-090b0d6576aa None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:01.594203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792b442-7485-460e-8733-090b0d6576aa None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:01.595558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 162/318] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:13:01.600561 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fa7cb-0c5c-497c-b98b-d584c90fabca None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:01.601024 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fa7cb-0c5c-497c-b98b-d584c90fabca None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:01.601483 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fa7cb-0c5c-497c-b98b-d584c90fabca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:01.602725 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e8646-5f75-404d-a682-86db6a28b2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:01.603323 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e8646-5f75-404d-a682-86db6a28b2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:01.603599 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e8646-5f75-404d-a682-86db6a28b2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:01.603872 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e8646-5f75-404d-a682-86db6a28b2a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:01.604594 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e8646-5f75-404d-a682-86db6a28b2a4 None admin] RBAC: Adding query filter params (role.id=3b1d23e3e45d44a3a8069b8e823410b5, scope.project.id=523fc40cf4044e598d36fb05a352d260, user.id=6ec6b5f76e9c4f16a5565ea1629b3563) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:01.604904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e8646-5f75-404d-a682-86db6a28b2a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:01.606372 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e8646-5f75-404d-a682-86db6a28b2a4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:01.612762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 157/319] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:01 2022] GET /identity/v3/role_assignments?user.id=6ec6b5f76e9c4f16a5565ea1629b3563&scope.project.id=523fc40cf4044e598d36fb05a352d260&role.id=3b1d23e3e45d44a3a8069b8e823410b5 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:01.621915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-bc2fa7cb-0c5c-497c-b98b-d584c90fabca None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:01.623521 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bc2fa7cb-0c5c-497c-b98b-d584c90fabca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:01.659023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:01.702816 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 163/320] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:01.725158 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e8646-5f75-404d-a682-86db6a28b2a4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:01.757110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:01.831839 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e8646-5f75-404d-a682-86db6a28b2a4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:01.869073 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e932c6d-b2ae-489a-aa59-850089a0c586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:01.869705 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e932c6d-b2ae-489a-aa59-850089a0c586 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:01.869985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e932c6d-b2ae-489a-aa59-850089a0c586 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:01.870281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e932c6d-b2ae-489a-aa59-850089a0c586 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:01.871428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e932c6d-b2ae-489a-aa59-850089a0c586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:01.872879 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e932c6d-b2ae-489a-aa59-850089a0c586 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:01.874489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 158/321] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:01.880175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fa7cb-0c5c-497c-b98b-d584c90fabca None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:01.985248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fa7cb-0c5c-497c-b98b-d584c90fabca None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:02.019549 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87ee16-8471-468e-a8e1-f68f0a10c923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:02.020152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87ee16-8471-468e-a8e1-f68f0a10c923 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:02.020424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87ee16-8471-468e-a8e1-f68f0a10c923 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:02.020690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87ee16-8471-468e-a8e1-f68f0a10c923 None admin] PATH_INFO: `/v3/groups` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:02.021307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87ee16-8471-468e-a8e1-f68f0a10c923 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:02.022758 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87ee16-8471-468e-a8e1-f68f0a10c923 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:02.044979 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 164/322] 10.222.0.39 () {60 vars in 1134 bytes} [Mon Aug 8 02:13:01 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:02.145198 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6f746-ba70-4146-828d-6d845821905e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:02.145536 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6f746-ba70-4146-828d-6d845821905e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:02.145756 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6f746-ba70-4146-828d-6d845821905e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:02.147418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 159/323] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:02.152826 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c9dbb-df38-4579-913d-1e7e5b4c79f1 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:02.153121 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c9dbb-df38-4579-913d-1e7e5b4c79f1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:02.153399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c9dbb-df38-4579-913d-1e7e5b4c79f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:02.172494 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-9a0c9dbb-df38-4579-913d-1e7e5b4c79f1 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:02.173733 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9a0c9dbb-df38-4579-913d-1e7e5b4c79f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:02.195418 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:02.256576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 165/324] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:02.287353 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6f746-ba70-4146-828d-6d845821905e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:02.288487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e67f5d-aedf-46ba-af88-321e4dea206d None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:02.288869 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e67f5d-aedf-46ba-af88-321e4dea206d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:02.289085 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e67f5d-aedf-46ba-af88-321e4dea206d None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:02.290525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 166/325] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:13:02.295858 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c7580-ff7e-4c5c-a56f-18523bd9b43e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:02.296135 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c7580-ff7e-4c5c-a56f-18523bd9b43e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:02.296407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c7580-ff7e-4c5c-a56f-18523bd9b43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:02.317541 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c86c7580-ff7e-4c5c-a56f-18523bd9b43e None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:02.318803 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c86c7580-ff7e-4c5c-a56f-18523bd9b43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:02.401079 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:02.402842 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 167/326] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:02.411316 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6f746-ba70-4146-828d-6d845821905e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:02.427440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c7580-ff7e-4c5c-a56f-18523bd9b43e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:02.457799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1afcfe-4bed-492b-8516-0142b3bb48d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:02.459003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1afcfe-4bed-492b-8516-0142b3bb48d9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:02.459294 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1afcfe-4bed-492b-8516-0142b3bb48d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:02.459569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1afcfe-4bed-492b-8516-0142b3bb48d9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:02.471216 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1afcfe-4bed-492b-8516-0142b3bb48d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:02.476665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-af1afcfe-4bed-492b-8516-0142b3bb48d9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:13:02.476665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:02.476665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:02.476665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:02.476665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:02.476665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:02.476665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:02.476665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:02.476665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:02.476665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:02.476665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:02.476665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:02.476665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:02.476665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:02.476665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:02.476665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:02.477525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:02.477525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:02.477525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:02.477525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:02.477525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:02.477525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:02.477525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:02.477525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:02.477525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:02.477525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:02.477525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:02.477525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:02.477525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:02.477525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:02.477525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:02.477525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:02.478379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:02.478379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:02.478379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:02.478379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:02.478379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:02.478379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:02.478379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:02.478379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:02.478379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:02.478379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:02.478379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:13:02.478379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:02.478379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 160/327] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:02.485711 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1afcfe-4bed-492b-8516-0142b3bb48d9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:02.570733 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c7580-ff7e-4c5c-a56f-18523bd9b43e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:02.582622 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:02.612225 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76220dd-48e9-4eb6-a40e-9e19547db581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:02.613202 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76220dd-48e9-4eb6-a40e-9e19547db581 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:02.613736 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76220dd-48e9-4eb6-a40e-9e19547db581 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:02.614198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76220dd-48e9-4eb6-a40e-9e19547db581 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:02.619684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1afcfe-4bed-492b-8516-0142b3bb48d9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:02.627029 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76220dd-48e9-4eb6-a40e-9e19547db581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:02.632136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-a76220dd-48e9-4eb6-a40e-9e19547db581 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:02.632136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:02.632136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:02.632136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:02.632136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:02.632136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:02.632136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:02.632136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:02.632136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:02.632136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:02.632136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:02.632136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:02.632136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:02.632136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:02.632136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:02.632136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:02.633008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:02.633008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:02.633008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:02.633008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:02.633008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:02.633008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:02.633008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:02.633008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:02.633008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:02.633008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:02.633008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:02.633008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:02.633008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:02.633008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:02.633008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:02.633008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:02.633778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:02.633778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:02.633778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:02.633778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:02.633778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:02.633778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:02.633778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:02.633778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:02.633778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:02.633778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:02.633778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:02.633778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:02.636371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 168/328] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:02.643975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76220dd-48e9-4eb6-a40e-9e19547db581 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:02.664899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6d307-0a9b-404d-a874-46af03955086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:02.665557 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6d307-0a9b-404d-a874-46af03955086 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:02.665944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6d307-0a9b-404d-a874-46af03955086 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:02.666229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6d307-0a9b-404d-a874-46af03955086 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:02.666938 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6d307-0a9b-404d-a874-46af03955086 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:02.667243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6d307-0a9b-404d-a874-46af03955086 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:02.668802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6d307-0a9b-404d-a874-46af03955086 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:02.680218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 161/329] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:02.687371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6d307-0a9b-404d-a874-46af03955086 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:02.709348 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:02.781664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76220dd-48e9-4eb6-a40e-9e19547db581 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:02.813985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6d307-0a9b-404d-a874-46af03955086 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:02.829142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431d92b-7493-45fe-8a9a-e92c9d646f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:02.830160 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431d92b-7493-45fe-8a9a-e92c9d646f2f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:02.830615 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431d92b-7493-45fe-8a9a-e92c9d646f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:02.830992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431d92b-7493-45fe-8a9a-e92c9d646f2f None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:02.832067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431d92b-7493-45fe-8a9a-e92c9d646f2f None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:02.832472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431d92b-7493-45fe-8a9a-e92c9d646f2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:02.834731 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431d92b-7493-45fe-8a9a-e92c9d646f2f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:02.846630 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 169/330] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:02.853634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431d92b-7493-45fe-8a9a-e92c9d646f2f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:02.859043 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec47b6a-597f-4ce9-b4dd-da4dbf819624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:02.859963 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec47b6a-597f-4ce9-b4dd-da4dbf819624 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:02.860501 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec47b6a-597f-4ce9-b4dd-da4dbf819624 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:02.860995 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec47b6a-597f-4ce9-b4dd-da4dbf819624 None admin] PATH_INFO: `/v3/users/4452f8b7b7ea4c71b682fb0744e92858` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:02.862394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec47b6a-597f-4ce9-b4dd-da4dbf819624 None admin] RBAC: Authorizing `identity:get_user(user_id=4452f8b7b7ea4c71b682fb0744e92858)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:02.863571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:02.864970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec47b6a-597f-4ce9-b4dd-da4dbf819624 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:02.866932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 162/331] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:02 2022] GET /identity/v3/users/4452f8b7b7ea4c71b682fb0744e92858 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:02.875658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec47b6a-597f-4ce9-b4dd-da4dbf819624 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:02.977064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:02.978946 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431d92b-7493-45fe-8a9a-e92c9d646f2f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:02.998098 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec47b6a-597f-4ce9-b4dd-da4dbf819624 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:03.026738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef9e51-558c-455c-a139-b707ec8f9398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:03.028067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef9e51-558c-455c-a139-b707ec8f9398 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:03.028067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef9e51-558c-455c-a139-b707ec8f9398 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:03.028067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef9e51-558c-455c-a139-b707ec8f9398 None admin] PATH_INFO: `/v3/users/76cb3c6199b140419ac3d66cfb278e04` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:03.029792 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef9e51-558c-455c-a139-b707ec8f9398 None admin] RBAC: Authorizing `identity:get_user(user_id=76cb3c6199b140419ac3d66cfb278e04)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:03.031152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef9e51-558c-455c-a139-b707ec8f9398 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:03.033141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 170/332] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:02 2022] GET /identity/v3/users/76cb3c6199b140419ac3d66cfb278e04 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:03.042821 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef9e51-558c-455c-a139-b707ec8f9398 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:03.046173 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31f765-0ae2-4fa4-a093-9e39d7b5f393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:03.047018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31f765-0ae2-4fa4-a093-9e39d7b5f393 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:03.047610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31f765-0ae2-4fa4-a093-9e39d7b5f393 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:03.048029 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31f765-0ae2-4fa4-a093-9e39d7b5f393 None admin] PATH_INFO: `/v3/system/users/4452f8b7b7ea4c71b682fb0744e92858/roles/3b1d23e3e45d44a3a8069b8e823410b5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:03.050757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31f765-0ae2-4fa4-a093-9e39d7b5f393 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4452f8b7b7ea4c71b682fb0744e92858, role_id=3b1d23e3e45d44a3a8069b8e823410b5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:03.052368 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31f765-0ae2-4fa4-a093-9e39d7b5f393 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:03.064224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 163/333] 10.222.0.39 () {58 vars in 1334 bytes} [Mon Aug 8 02:13:02 2022] PUT /identity/v3/system/users/4452f8b7b7ea4c71b682fb0744e92858/roles/3b1d23e3e45d44a3a8069b8e823410b5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:03.139401 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:03.174267 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef9e51-558c-455c-a139-b707ec8f9398 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:03.219173 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81611b7e-19c9-47e6-85b1-5b44a2ad6c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:03.219792 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81611b7e-19c9-47e6-85b1-5b44a2ad6c96 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:03.220067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81611b7e-19c9-47e6-85b1-5b44a2ad6c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:03.220339 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81611b7e-19c9-47e6-85b1-5b44a2ad6c96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:03.221135 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81611b7e-19c9-47e6-85b1-5b44a2ad6c96 None admin] RBAC: Adding query filter params (role.id=42c846f6944b40c8af5eeb8360cb7c7a, user.id=76cb3c6199b140419ac3d66cfb278e04, scope.system=all) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:03.221448 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81611b7e-19c9-47e6-85b1-5b44a2ad6c96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:03.222987 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81611b7e-19c9-47e6-85b1-5b44a2ad6c96 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:03.236728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 171/334] 10.222.0.39 () {56 vars in 1308 bytes} [Mon Aug 8 02:13:03 2022] GET /identity/v3/role_assignments?user.id=76cb3c6199b140419ac3d66cfb278e04&scope.system=all&role.id=42c846f6944b40c8af5eeb8360cb7c7a => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:04.858957 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485780a7-6394-4d86-9b1c-dcb0e0c41ef8 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:04.859468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485780a7-6394-4d86-9b1c-dcb0e0c41ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:04.859468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485780a7-6394-4d86-9b1c-dcb0e0c41ef8 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:04.861050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 164/335] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:04.867144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161d8f7-fa77-40d2-af0d-0b2d427c7b4e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:04.867560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161d8f7-fa77-40d2-af0d-0b2d427c7b4e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:04.867746 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161d8f7-fa77-40d2-af0d-0b2d427c7b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:04.895269 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7161d8f7-fa77-40d2-af0d-0b2d427c7b4e None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:04.897010 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7161d8f7-fa77-40d2-af0d-0b2d427c7b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:04.986889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 172/336] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:05.022406 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485780a7-6394-4d86-9b1c-dcb0e0c41ef8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:05.055962 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb840d-7fba-488c-8c06-7acfca4f8901 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:05.056254 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb840d-7fba-488c-8c06-7acfca4f8901 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:05.056512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb840d-7fba-488c-8c06-7acfca4f8901 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:05.057992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 173/337] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:13:05.066938 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1055326-ba55-4de6-a191-2e28283cd177 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:05.067571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1055326-ba55-4de6-a191-2e28283cd177 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:05.068626 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1055326-ba55-4de6-a191-2e28283cd177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:05.103405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b1055326-ba55-4de6-a191-2e28283cd177 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:05.104969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b1055326-ba55-4de6-a191-2e28283cd177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:05.219989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485780a7-6394-4d86-9b1c-dcb0e0c41ef8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:05.226605 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:05.242528 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 174/338] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:05.283162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1055326-ba55-4de6-a191-2e28283cd177 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:05.286958 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22326346-4150-4d4e-9f78-e9fe4e2f914a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:05.287594 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22326346-4150-4d4e-9f78-e9fe4e2f914a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:05.287871 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22326346-4150-4d4e-9f78-e9fe4e2f914a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:05.288136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22326346-4150-4d4e-9f78-e9fe4e2f914a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:05.307089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22326346-4150-4d4e-9f78-e9fe4e2f914a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:05.312549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-22326346-4150-4d4e-9f78-e9fe4e2f914a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:05.312549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:05.312549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:05.312549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:05.312549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:05.312549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:05.312549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:05.312549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:05.312549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:05.312549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:05.312549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:05.312549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:05.312549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:05.312549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:05.312549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:05.312549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:05.313430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:05.313430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:05.313430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:05.313430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:05.313430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:05.313430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:05.313430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:05.313430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:05.313430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:05.313430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:05.313430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:05.313430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:05.313430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:05.313430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:05.313430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:05.313430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:05.314182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:05.314182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:05.314182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:05.314182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:05.314182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:05.314182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:05.314182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:05.314182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:05.314182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:05.314182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:05.314182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:05.314182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:05.314182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 165/339] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:13:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:05.320037 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22326346-4150-4d4e-9f78-e9fe4e2f914a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:05.344436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:05.516027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1055326-ba55-4de6-a191-2e28283cd177 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:05.544810 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22326346-4150-4d4e-9f78-e9fe4e2f914a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:05.584912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaa125-06af-4edd-9b1d-b747c5f50d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:05.585528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaa125-06af-4edd-9b1d-b747c5f50d3c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:05.585808 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaa125-06af-4edd-9b1d-b747c5f50d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:05.586078 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaa125-06af-4edd-9b1d-b747c5f50d3c None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:05.589052 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceaa125-06af-4edd-9b1d-b747c5f50d3c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:05.589468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceaa125-06af-4edd-9b1d-b747c5f50d3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:05.590923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceaa125-06af-4edd-9b1d-b747c5f50d3c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:05.600097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ab15a-56a1-4bc4-9223-ae5fade8a288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:05.600799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 166/340] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:05.601058 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ab15a-56a1-4bc4-9223-ae5fade8a288 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:05.601349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ab15a-56a1-4bc4-9223-ae5fade8a288 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:05.601616 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ab15a-56a1-4bc4-9223-ae5fade8a288 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:05.606265 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaa125-06af-4edd-9b1d-b747c5f50d3c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:05.626766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ab15a-56a1-4bc4-9223-ae5fade8a288 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:05.631576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-9a3ab15a-56a1-4bc4-9223-ae5fade8a288 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:05.631576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:05.631576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:05.631576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:05.631576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:05.631576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:05.631576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:05.631576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:05.631576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:05.631576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:05.631576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:05.631576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:05.631576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:05.631576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:05.631576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:05.631576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:05.632446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:05.632446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:05.632446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:05.632446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:05.632446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:05.632446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:05.632446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:05.632446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:05.632446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:05.632446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:05.632446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:05.632446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:05.632446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:05.632446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:05.632446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:05.632446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:05.633237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:05.633237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:05.633237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:05.633237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:05.633237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:05.633237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:05.633237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:05.633237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:05.633237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:05.633237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:05.633237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:05.633237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:05.633237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 175/341] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:05.641547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ab15a-56a1-4bc4-9223-ae5fade8a288 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:05.749961 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:05.752021 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:05.810315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaa125-06af-4edd-9b1d-b747c5f50d3c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:05.820560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ab15a-56a1-4bc4-9223-ae5fade8a288 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:05.869060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a11c01-c640-4b19-996b-f4d4c8336c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:05.869677 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a11c01-c640-4b19-996b-f4d4c8336c55 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:05.869935 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a11c01-c640-4b19-996b-f4d4c8336c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:05.870207 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a11c01-c640-4b19-996b-f4d4c8336c55 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:05.870894 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a11c01-c640-4b19-996b-f4d4c8336c55 None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:05.871197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a11c01-c640-4b19-996b-f4d4c8336c55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:05.872720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a11c01-c640-4b19-996b-f4d4c8336c55 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:05.885355 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 176/342] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:05.893555 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a11c01-c640-4b19-996b-f4d4c8336c55 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:05.893720 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d0c55-f6c7-4b17-b6a1-a3b6b393e968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:05.894381 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d0c55-f6c7-4b17-b6a1-a3b6b393e968 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:05.894637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d0c55-f6c7-4b17-b6a1-a3b6b393e968 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:05.894901 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d0c55-f6c7-4b17-b6a1-a3b6b393e968 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:06.010027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a11c01-c640-4b19-996b-f4d4c8336c55 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:06.025525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:06.038831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d0c55-f6c7-4b17-b6a1-a3b6b393e968 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:06.040287 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d0c55-f6c7-4b17-b6a1-a3b6b393e968 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:06.048077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a64de-e43a-40da-b6d3-5b942c99d61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:06.048722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a64de-e43a-40da-b6d3-5b942c99d61e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:06.048972 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a64de-e43a-40da-b6d3-5b942c99d61e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:06.049246 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a64de-e43a-40da-b6d3-5b942c99d61e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:06.124074 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a64de-e43a-40da-b6d3-5b942c99d61e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:06.125599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a64de-e43a-40da-b6d3-5b942c99d61e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:06.188030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:06.230944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 167/343] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:06.233177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23beb99b-9883-4a27-b607-7dacbb6b5fbe None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:06.233457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23beb99b-9883-4a27-b607-7dacbb6b5fbe None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:06.233726 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23beb99b-9883-4a27-b607-7dacbb6b5fbe None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:06.236099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 168/344] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:06.237207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be526a-e3e0-473c-a324-dbb9e3e883f9 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:06.237488 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be526a-e3e0-473c-a324-dbb9e3e883f9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:06.237754 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be526a-e3e0-473c-a324-dbb9e3e883f9 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:06.240884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 169/345] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:06.241577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 177/346] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 02:13:06.242651 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be526a-e3e0-473c-a324-dbb9e3e883f9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:06.244737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468b3ee-4cdb-47b8-9a4a-ebfab16dff8b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:06.245021 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468b3ee-4cdb-47b8-9a4a-ebfab16dff8b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:06.245297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468b3ee-4cdb-47b8-9a4a-ebfab16dff8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:06.266109 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-4468b3ee-4cdb-47b8-9a4a-ebfab16dff8b None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:06.267582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4468b3ee-4cdb-47b8-9a4a-ebfab16dff8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:06.279076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:06.367493 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 178/347] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:06.368624 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468b3ee-4cdb-47b8-9a4a-ebfab16dff8b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:06.381262 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be526a-e3e0-473c-a324-dbb9e3e883f9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:06.417309 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6af7be-71ca-4052-9be2-25c4a9493c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:06.417772 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6af7be-71ca-4052-9be2-25c4a9493c02 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:06.418131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6af7be-71ca-4052-9be2-25c4a9493c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:06.418444 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6af7be-71ca-4052-9be2-25c4a9493c02 None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:06.419621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6af7be-71ca-4052-9be2-25c4a9493c02 None admin] RBAC: Authorizing `identity:get_project(project_id=3441a04d36074741b13ebfa058e656b8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:06.422292 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6af7be-71ca-4052-9be2-25c4a9493c02 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:06.423459 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 170/348] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:06 2022] GET /identity/v3/projects/3441a04d36074741b13ebfa058e656b8 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:06.424276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804b237-f85a-402c-bf8d-eb9ae14c03f0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:06.424553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804b237-f85a-402c-bf8d-eb9ae14c03f0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:06.424822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804b237-f85a-402c-bf8d-eb9ae14c03f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:06.447294 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e804b237-f85a-402c-bf8d-eb9ae14c03f0 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:06.448505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e804b237-f85a-402c-bf8d-eb9ae14c03f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:06.485091 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:06.497190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468b3ee-4cdb-47b8-9a4a-ebfab16dff8b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:06.531693 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 171/349] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:06.532767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79479a06-46b8-4ba3-900d-ae2d3fcabee8 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:06.533048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79479a06-46b8-4ba3-900d-ae2d3fcabee8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:06.533310 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79479a06-46b8-4ba3-900d-ae2d3fcabee8 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:06.538352 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 172/350] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:06.539588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79479a06-46b8-4ba3-900d-ae2d3fcabee8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:06.561195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a14085-e3b1-4067-87ee-f61378caff85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:06.562387 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a14085-e3b1-4067-87ee-f61378caff85 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:06.564340 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a14085-e3b1-4067-87ee-f61378caff85 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:06.565592 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a14085-e3b1-4067-87ee-f61378caff85 None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:06.569794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a14085-e3b1-4067-87ee-f61378caff85 None admin] RBAC: Authorizing `identity:get_project(project_id=3441a04d36074741b13ebfa058e656b8)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:06.571332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a14085-e3b1-4067-87ee-f61378caff85 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:06.573014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 179/351] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:06 2022] GET /identity/v3/projects/3441a04d36074741b13ebfa058e656b8 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:06.573775 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a14085-e3b1-4067-87ee-f61378caff85 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:06.590347 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:06.680781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79479a06-46b8-4ba3-900d-ae2d3fcabee8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:06.706041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a14085-e3b1-4067-87ee-f61378caff85 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:06.729185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094379b6-c4ed-42d4-9a6a-da53b47594d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:06.729968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094379b6-c4ed-42d4-9a6a-da53b47594d6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:06.730238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094379b6-c4ed-42d4-9a6a-da53b47594d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:06.730531 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094379b6-c4ed-42d4-9a6a-da53b47594d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:06.749847 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094379b6-c4ed-42d4-9a6a-da53b47594d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:06.755914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-094379b6-c4ed-42d4-9a6a-da53b47594d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:06.755914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:06.755914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:06.755914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:06.755914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:06.755914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:06.755914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:06.755914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:06.755914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:06.755914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:06.755914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:06.755914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:06.755914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:06.755914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:06.755914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:06.755914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:06.756844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:06.756844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:06.756844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:06.756844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:06.756844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:06.756844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:06.756844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:06.756844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:06.756844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:06.756844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:06.756844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:06.756844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:06.756844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:06.756844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:06.756844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:06.756844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:06.757621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:06.757621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:06.757621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:06.757621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:06.757621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:06.757621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:06.757621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:06.757621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:06.757621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:06.757621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:06.757621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:06.757621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:06.757621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 173/352] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:06.758289 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9eabbf-aa39-4e39-84fd-49e4e0453137 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:06.760575 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9eabbf-aa39-4e39-84fd-49e4e0453137 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:06.760575 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9eabbf-aa39-4e39-84fd-49e4e0453137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:06.764648 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c9b83-9a0b-4223-89a7-41e2e7eff03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:06.765291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c9b83-9a0b-4223-89a7-41e2e7eff03c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:06.765574 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c9b83-9a0b-4223-89a7-41e2e7eff03c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:06.765841 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c9b83-9a0b-4223-89a7-41e2e7eff03c None admin] PATH_INFO: `/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:06.773179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c9b83-9a0b-4223-89a7-41e2e7eff03c None admin] RBAC: Authorizing `identity:get_group(group_id=c1f75f60b9314d738b2ac7b030ea42d3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:06.775045 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c9b83-9a0b-4223-89a7-41e2e7eff03c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:06.776679 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 180/353] 10.222.0.39 () {56 vars in 1180 bytes} [Mon Aug 8 02:13:06 2022] GET /identity/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:06.778062 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c9b83-9a0b-4223-89a7-41e2e7eff03c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:06.778190 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1b9eabbf-aa39-4e39-84fd-49e4e0453137 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:06.779354 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1b9eabbf-aa39-4e39-84fd-49e4e0453137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:06.808254 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:06.862076 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 174/354] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:06.862981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9eabbf-aa39-4e39-84fd-49e4e0453137 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:06.875737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:06.905708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c9b83-9a0b-4223-89a7-41e2e7eff03c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:06.945518 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5e0ed-86f1-4b17-8bcb-06257b8f6df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:06.946514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5e0ed-86f1-4b17-8bcb-06257b8f6df8 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:06.946951 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5e0ed-86f1-4b17-8bcb-06257b8f6df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:06.947227 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5e0ed-86f1-4b17-8bcb-06257b8f6df8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:06.963073 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5e0ed-86f1-4b17-8bcb-06257b8f6df8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:06.968710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-0cf5e0ed-86f1-4b17-8bcb-06257b8f6df8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:13:06.968710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:06.968710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:06.968710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:06.968710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:06.968710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:06.968710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:06.968710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:06.968710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:06.968710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:06.968710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:06.968710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:06.968710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:06.968710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:06.968710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:06.968710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:06.969566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:06.969566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:06.969566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:06.969566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:06.969566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:06.969566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:06.969566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:06.969566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:06.969566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:06.969566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:06.969566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:06.969566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:06.969566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:06.969566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:06.969566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:06.969566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:06.970393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:06.970393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:06.970393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:06.970393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:06.970393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:06.970393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:06.970393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:06.970393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:06.970393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:06.970393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:06.970393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:13:06.970393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:06.970393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 181/355] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:06.970393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf24e0-677e-41a6-abde-29f8e5eac7b3 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:06.971081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf24e0-677e-41a6-abde-29f8e5eac7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:06.971081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf24e0-677e-41a6-abde-29f8e5eac7b3 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:06.974685 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 182/356] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:06.974685 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf24e0-677e-41a6-abde-29f8e5eac7b3 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:06.978413 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9eabbf-aa39-4e39-84fd-49e4e0453137 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:07.017018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81bd9c-a4c4-4942-b723-163af06d7efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:07.018015 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81bd9c-a4c4-4942-b723-163af06d7efd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:07.018015 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81bd9c-a4c4-4942-b723-163af06d7efd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:07.018187 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81bd9c-a4c4-4942-b723-163af06d7efd None admin] PATH_INFO: `/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:07.019394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81bd9c-a4c4-4942-b723-163af06d7efd None admin] RBAC: Authorizing `identity:get_group(group_id=c1f75f60b9314d738b2ac7b030ea42d3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:07.021690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81bd9c-a4c4-4942-b723-163af06d7efd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:07.023470 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 175/357] 10.222.0.39 () {56 vars in 1180 bytes} [Mon Aug 8 02:13:06 2022] GET /identity/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:07.024371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81bd9c-a4c4-4942-b723-163af06d7efd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:07.057353 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:07.100336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf24e0-677e-41a6-abde-29f8e5eac7b3 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:07.140815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226da516-2350-46d1-8a25-98f2ac1424d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:07.141424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226da516-2350-46d1-8a25-98f2ac1424d9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:07.141708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226da516-2350-46d1-8a25-98f2ac1424d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:07.141977 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226da516-2350-46d1-8a25-98f2ac1424d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:07.142702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226da516-2350-46d1-8a25-98f2ac1424d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:07.143007 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226da516-2350-46d1-8a25-98f2ac1424d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:07.143784 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81bd9c-a4c4-4942-b723-163af06d7efd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:07.144380 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226da516-2350-46d1-8a25-98f2ac1424d9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:07.147466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:07.155962 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 183/358] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:07.157106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226da516-2350-46d1-8a25-98f2ac1424d9 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:07.185140 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c638f3-d524-40d5-a79b-b02c415bd2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:07.185758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c638f3-d524-40d5-a79b-b02c415bd2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:07.186025 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c638f3-d524-40d5-a79b-b02c415bd2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:07.186362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c638f3-d524-40d5-a79b-b02c415bd2d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:07.187085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c638f3-d524-40d5-a79b-b02c415bd2d0 None admin] RBAC: Adding query filter params (group.id=c1f75f60b9314d738b2ac7b030ea42d3, role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839, scope.project.id=3441a04d36074741b13ebfa058e656b8) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:07.187394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c638f3-d524-40d5-a79b-b02c415bd2d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:07.188812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c638f3-d524-40d5-a79b-b02c415bd2d0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:07.195703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 176/359] 10.222.0.39 () {56 vars in 1376 bytes} [Mon Aug 8 02:13:07 2022] GET /identity/v3/role_assignments?group.id=c1f75f60b9314d738b2ac7b030ea42d3&scope.project.id=3441a04d36074741b13ebfa058e656b8&role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:07.196232 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c638f3-d524-40d5-a79b-b02c415bd2d0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:07.291212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226da516-2350-46d1-8a25-98f2ac1424d9 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:07.322464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:07.335490 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c638f3-d524-40d5-a79b-b02c415bd2d0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:07.336316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa95c0-df8a-4eee-a76a-23d2e627c949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:07.337897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa95c0-df8a-4eee-a76a-23d2e627c949 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:07.338957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa95c0-df8a-4eee-a76a-23d2e627c949 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:07.338957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa95c0-df8a-4eee-a76a-23d2e627c949 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:07.356939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa95c0-df8a-4eee-a76a-23d2e627c949 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:07.363186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-a1aa95c0-df8a-4eee-a76a-23d2e627c949 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:07.363186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:07.363186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:07.363186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:07.363186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:07.363186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:07.363186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:07.363186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:07.363186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:07.363186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:07.363186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:07.363186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:07.363186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:07.363186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:07.363186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:07.363186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:07.364049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:07.364049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:07.364049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:07.364049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:07.364049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:07.364049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:07.364049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:07.364049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:07.364049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:07.364049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:07.364049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:07.364049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:07.364049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:07.364049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:07.364049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:07.364049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:07.365022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:07.365022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:07.365022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:07.365022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:07.365022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:07.365022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:07.365022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:07.365022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:07.365022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:07.365022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:07.365022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:07.365022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:07.367882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 184/360] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:13:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:07.370258 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b68c92-c799-46b0-beec-83f2d8b3f5bb None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:07.371023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b68c92-c799-46b0-beec-83f2d8b3f5bb None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:07.371439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b68c92-c799-46b0-beec-83f2d8b3f5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:07.379514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c03af-74f5-45b7-a4d7-723d2805d316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:07.380885 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c03af-74f5-45b7-a4d7-723d2805d316 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:07.381158 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c03af-74f5-45b7-a4d7-723d2805d316 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:07.381421 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c03af-74f5-45b7-a4d7-723d2805d316 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:07.382076 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c03af-74f5-45b7-a4d7-723d2805d316 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:07.383412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c03af-74f5-45b7-a4d7-723d2805d316 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:07.384794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c03af-74f5-45b7-a4d7-723d2805d316 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:07.391904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-98b68c92-c799-46b0-beec-83f2d8b3f5bb None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:07.393065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-98b68c92-c799-46b0-beec-83f2d8b3f5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:07.394121 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 177/361] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:07.395039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c03af-74f5-45b7-a4d7-723d2805d316 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:07.467990 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:07.491897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 185/362] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:07.493492 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b68c92-c799-46b0-beec-83f2d8b3f5bb None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:07.525347 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c03af-74f5-45b7-a4d7-723d2805d316 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:07.576495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e46261-3171-4750-9c86-b94e64773c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:07.576495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e46261-3171-4750-9c86-b94e64773c26 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:07.576495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e46261-3171-4750-9c86-b94e64773c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:07.576495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e46261-3171-4750-9c86-b94e64773c26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:07.576855 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e46261-3171-4750-9c86-b94e64773c26 None admin] RBAC: Adding query filter params (group.id=c1f75f60b9314d738b2ac7b030ea42d3, role.id=42c846f6944b40c8af5eeb8360cb7c7a, scope.project.id=3441a04d36074741b13ebfa058e656b8) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:07.576855 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e46261-3171-4750-9c86-b94e64773c26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:07.581166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e46261-3171-4750-9c86-b94e64773c26 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:07.590071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 178/363] 10.222.0.39 () {56 vars in 1376 bytes} [Mon Aug 8 02:13:07 2022] GET /identity/v3/role_assignments?group.id=c1f75f60b9314d738b2ac7b030ea42d3&scope.project.id=3441a04d36074741b13ebfa058e656b8&role.id=42c846f6944b40c8af5eeb8360cb7c7a => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:07.591067 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e46261-3171-4750-9c86-b94e64773c26 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:07.613756 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:07.639920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b68c92-c799-46b0-beec-83f2d8b3f5bb None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:07.681644 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd528232-483a-4456-9035-fa38351e3453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:07.682541 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd528232-483a-4456-9035-fa38351e3453 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:07.683054 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd528232-483a-4456-9035-fa38351e3453 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:07.683485 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd528232-483a-4456-9035-fa38351e3453 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:07.722944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e46261-3171-4750-9c86-b94e64773c26 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:07.768174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7388a38-bae5-4f0a-be8b-7b313b8717d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:07.769087 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7388a38-bae5-4f0a-be8b-7b313b8717d5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:07.769508 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7388a38-bae5-4f0a-be8b-7b313b8717d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:07.769918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7388a38-bae5-4f0a-be8b-7b313b8717d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:07.770830 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7388a38-bae5-4f0a-be8b-7b313b8717d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:07.771284 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7388a38-bae5-4f0a-be8b-7b313b8717d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:07.771664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:07.774193 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7388a38-bae5-4f0a-be8b-7b313b8717d5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:07.784163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 179/364] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:07.785244 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7388a38-bae5-4f0a-be8b-7b313b8717d5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:07.788450 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd528232-483a-4456-9035-fa38351e3453 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:07.790897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd528232-483a-4456-9035-fa38351e3453 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:07.878330 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:07.908665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7388a38-bae5-4f0a-be8b-7b313b8717d5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:07.922762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 186/365] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:07.923657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd528232-483a-4456-9035-fa38351e3453 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:07.957780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cc5de-a13d-4b96-99e7-8b9c88744fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:07.958964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cc5de-a13d-4b96-99e7-8b9c88744fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:07.959802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cc5de-a13d-4b96-99e7-8b9c88744fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:07.960218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cc5de-a13d-4b96-99e7-8b9c88744fd5 None admin] PATH_INFO: `/v3/users/4452f8b7b7ea4c71b682fb0744e92858` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:07.962034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2cc5de-a13d-4b96-99e7-8b9c88744fd5 None admin] RBAC: Authorizing `identity:get_user(user_id=4452f8b7b7ea4c71b682fb0744e92858)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:07.964351 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2cc5de-a13d-4b96-99e7-8b9c88744fd5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:07.967626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 180/366] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:07 2022] GET /identity/v3/users/4452f8b7b7ea4c71b682fb0744e92858 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:07.968601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cc5de-a13d-4b96-99e7-8b9c88744fd5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:08.030504 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:08.044534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd528232-483a-4456-9035-fa38351e3453 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:08.088517 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34443073-c2ad-4003-906e-0d192393e338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:08.089151 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34443073-c2ad-4003-906e-0d192393e338 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:08.089428 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34443073-c2ad-4003-906e-0d192393e338 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:08.089700 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34443073-c2ad-4003-906e-0d192393e338 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:08.105033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cc5de-a13d-4b96-99e7-8b9c88744fd5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:08.105529 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34443073-c2ad-4003-906e-0d192393e338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:08.109892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-34443073-c2ad-4003-906e-0d192393e338 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:08.109892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:08.109892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:08.109892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:08.109892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:08.109892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:08.109892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:08.109892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:08.109892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:08.109892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:08.109892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:08.109892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:08.109892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:08.109892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:08.109892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:08.109892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:08.110956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:08.110956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:08.110956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:08.110956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:08.110956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:08.110956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:08.110956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:08.110956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:08.110956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:08.110956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:08.110956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:08.110956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:08.110956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:08.110956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:08.110956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:08.110956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:08.111784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:08.111784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:08.111784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:08.111784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:08.111784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:08.111784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:08.111784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:08.111784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:08.111784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:08.111784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:08.111784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:08.111784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:08.112832 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 187/367] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:08.114235 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34443073-c2ad-4003-906e-0d192393e338 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:08.157873 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170ee9f-8eab-45f7-bef2-beafda57658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:08.158742 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170ee9f-8eab-45f7-bef2-beafda57658c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:08.159159 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170ee9f-8eab-45f7-bef2-beafda57658c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:08.159800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170ee9f-8eab-45f7-bef2-beafda57658c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:08.183370 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:08.244574 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34443073-c2ad-4003-906e-0d192393e338 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:08.260540 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170ee9f-8eab-45f7-bef2-beafda57658c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:08.262136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170ee9f-8eab-45f7-bef2-beafda57658c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:08.280061 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657fc21-e8c5-4639-a72b-0c45a2e519d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:08.280999 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657fc21-e8c5-4639-a72b-0c45a2e519d7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:08.281220 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657fc21-e8c5-4639-a72b-0c45a2e519d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:08.281868 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657fc21-e8c5-4639-a72b-0c45a2e519d7 None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:08.283035 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657fc21-e8c5-4639-a72b-0c45a2e519d7 None admin] RBAC: Authorizing `identity:get_project(project_id=523fc40cf4044e598d36fb05a352d260)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:08.285007 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657fc21-e8c5-4639-a72b-0c45a2e519d7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:08.287486 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 188/368] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:08 2022] GET /identity/v3/projects/523fc40cf4044e598d36fb05a352d260 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:08.288897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657fc21-e8c5-4639-a72b-0c45a2e519d7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:08.317439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:08.389588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 181/369] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:08.390472 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170ee9f-8eab-45f7-bef2-beafda57658c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:08.431792 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657fc21-e8c5-4639-a72b-0c45a2e519d7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:08.454636 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:08.477746 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adde315-7753-4717-b8ac-897c61b9c6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:08.478617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adde315-7753-4717-b8ac-897c61b9c6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:08.479084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adde315-7753-4717-b8ac-897c61b9c6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:08.479493 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adde315-7753-4717-b8ac-897c61b9c6a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:08.480387 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adde315-7753-4717-b8ac-897c61b9c6a7 None admin] RBAC: Adding query filter params (role.id=3b1d23e3e45d44a3a8069b8e823410b5, user.id=4452f8b7b7ea4c71b682fb0744e92858, scope.system=all) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:08.480837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adde315-7753-4717-b8ac-897c61b9c6a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:08.482504 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adde315-7753-4717-b8ac-897c61b9c6a7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:08.496020 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 189/370] 10.222.0.39 () {56 vars in 1308 bytes} [Mon Aug 8 02:13:08 2022] GET /identity/v3/role_assignments?user.id=4452f8b7b7ea4c71b682fb0744e92858&scope.system=all&role.id=3b1d23e3e45d44a3a8069b8e823410b5 => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:08.498202 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adde315-7753-4717-b8ac-897c61b9c6a7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:08.523053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170ee9f-8eab-45f7-bef2-beafda57658c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:08.581039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736bc4e8-a774-42f3-aa4a-db28ea1e2fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:08.581586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bc4e8-a774-42f3-aa4a-db28ea1e2fed None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:08.581866 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bc4e8-a774-42f3-aa4a-db28ea1e2fed None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:08.582138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bc4e8-a774-42f3-aa4a-db28ea1e2fed None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:08.583009 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736bc4e8-a774-42f3-aa4a-db28ea1e2fed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:08.586304 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736bc4e8-a774-42f3-aa4a-db28ea1e2fed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:08.586304 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736bc4e8-a774-42f3-aa4a-db28ea1e2fed None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:08.598482 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 182/371] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:08.599945 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736bc4e8-a774-42f3-aa4a-db28ea1e2fed None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:08.619225 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:08.676374 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adde315-7753-4717-b8ac-897c61b9c6a7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:08.729566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbe1a5-b68b-40b6-b5f7-aa4d777ef03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:08.730619 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbe1a5-b68b-40b6-b5f7-aa4d777ef03c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:08.730969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbe1a5-b68b-40b6-b5f7-aa4d777ef03c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:08.731311 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbe1a5-b68b-40b6-b5f7-aa4d777ef03c None admin] PATH_INFO: `/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:08.732675 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbe1a5-b68b-40b6-b5f7-aa4d777ef03c None admin] RBAC: Authorizing `identity:get_group(group_id=c1f75f60b9314d738b2ac7b030ea42d3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:08.734890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbe1a5-b68b-40b6-b5f7-aa4d777ef03c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:08.737167 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 190/372] 10.222.0.39 () {56 vars in 1180 bytes} [Mon Aug 8 02:13:08 2022] GET /identity/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:08.737853 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbe1a5-b68b-40b6-b5f7-aa4d777ef03c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:08.758337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736bc4e8-a774-42f3-aa4a-db28ea1e2fed None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:08.774299 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:08.807504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948fee9-e382-4a39-a308-0180223cdeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:08.808419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948fee9-e382-4a39-a308-0180223cdeb0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:08.808868 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948fee9-e382-4a39-a308-0180223cdeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:08.809448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948fee9-e382-4a39-a308-0180223cdeb0 None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:08.811074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948fee9-e382-4a39-a308-0180223cdeb0 None admin] RBAC: Authorizing `identity:get_project(project_id=523fc40cf4044e598d36fb05a352d260)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:08.812914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948fee9-e382-4a39-a308-0180223cdeb0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:08.814981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 183/373] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:08 2022] GET /identity/v3/projects/523fc40cf4044e598d36fb05a352d260 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:08.818288 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948fee9-e382-4a39-a308-0180223cdeb0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:08.878518 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbe1a5-b68b-40b6-b5f7-aa4d777ef03c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:08.927662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974ff0b-fa2e-4ccd-bb50-eea971639632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:08.928297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974ff0b-fa2e-4ccd-bb50-eea971639632 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:08.928553 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974ff0b-fa2e-4ccd-bb50-eea971639632 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:08.928830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974ff0b-fa2e-4ccd-bb50-eea971639632 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:08.947421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:08.962566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948fee9-e382-4a39-a308-0180223cdeb0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:09.003599 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5887c1-0b12-4c25-8a61-f76a1630498c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:09.005054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5887c1-0b12-4c25-8a61-f76a1630498c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:09.006398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5887c1-0b12-4c25-8a61-f76a1630498c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:09.006893 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5887c1-0b12-4c25-8a61-f76a1630498c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:09.008452 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5887c1-0b12-4c25-8a61-f76a1630498c None admin] RBAC: Adding query filter params (group.id=c1f75f60b9314d738b2ac7b030ea42d3, role.id=42c846f6944b40c8af5eeb8360cb7c7a, scope.project.id=523fc40cf4044e598d36fb05a352d260) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:09.009231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5887c1-0b12-4c25-8a61-f76a1630498c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:09.011144 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5887c1-0b12-4c25-8a61-f76a1630498c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:09.019103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 184/374] 10.222.0.39 () {56 vars in 1376 bytes} [Mon Aug 8 02:13:08 2022] GET /identity/v3/role_assignments?group.id=c1f75f60b9314d738b2ac7b030ea42d3&scope.project.id=523fc40cf4044e598d36fb05a352d260&role.id=42c846f6944b40c8af5eeb8360cb7c7a => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:09.020198 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5887c1-0b12-4c25-8a61-f76a1630498c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:09.023637 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974ff0b-fa2e-4ccd-bb50-eea971639632 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:09.026238 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974ff0b-fa2e-4ccd-bb50-eea971639632 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:09.084455 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:09.156206 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5887c1-0b12-4c25-8a61-f76a1630498c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:09.157145 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 191/375] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:09.164611 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974ff0b-fa2e-4ccd-bb50-eea971639632 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:09.215205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30536918-8a8b-454f-8470-4f1718ebe72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:09.216375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30536918-8a8b-454f-8470-4f1718ebe72e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:09.217700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30536918-8a8b-454f-8470-4f1718ebe72e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:09.218194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30536918-8a8b-454f-8470-4f1718ebe72e None admin] PATH_INFO: `/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:09.219671 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30536918-8a8b-454f-8470-4f1718ebe72e None admin] RBAC: Authorizing `identity:get_group(group_id=c1f75f60b9314d738b2ac7b030ea42d3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:09.221375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30536918-8a8b-454f-8470-4f1718ebe72e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:09.224634 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 185/376] 10.222.0.39 () {56 vars in 1180 bytes} [Mon Aug 8 02:13:09 2022] GET /identity/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:09.235304 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30536918-8a8b-454f-8470-4f1718ebe72e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:09.235963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:09.325424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974ff0b-fa2e-4ccd-bb50-eea971639632 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:09.362948 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba5dce3-b20b-479f-9963-07ed14a63867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:09.363974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5dce3-b20b-479f-9963-07ed14a63867 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:09.365099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5dce3-b20b-479f-9963-07ed14a63867 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:09.365584 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5dce3-b20b-479f-9963-07ed14a63867 None admin] PATH_INFO: `/v3/projects/520cd26c01a04fe5b710bc3899ff38a8` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:09.367072 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba5dce3-b20b-479f-9963-07ed14a63867 None admin] RBAC: Authorizing `identity:get_project(project_id=520cd26c01a04fe5b710bc3899ff38a8)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:09.368858 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba5dce3-b20b-479f-9963-07ed14a63867 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:09.372440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 192/377] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:09 2022] GET /identity/v3/projects/520cd26c01a04fe5b710bc3899ff38a8 => generated 344 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:09.378181 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba5dce3-b20b-479f-9963-07ed14a63867 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:09.398204 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30536918-8a8b-454f-8470-4f1718ebe72e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:09.408365 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:09.461892 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04090d-a392-48ec-85f6-4ef73940955a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:09.462530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04090d-a392-48ec-85f6-4ef73940955a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:09.462807 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04090d-a392-48ec-85f6-4ef73940955a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:09.463073 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04090d-a392-48ec-85f6-4ef73940955a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:09.463798 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a04090d-a392-48ec-85f6-4ef73940955a None admin] RBAC: Adding query filter params (group.id=c1f75f60b9314d738b2ac7b030ea42d3, role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839, scope.project.id=523fc40cf4044e598d36fb05a352d260) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:09.464108 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a04090d-a392-48ec-85f6-4ef73940955a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:09.465543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a04090d-a392-48ec-85f6-4ef73940955a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:09.479690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 186/378] 10.222.0.39 () {56 vars in 1376 bytes} [Mon Aug 8 02:13:09 2022] GET /identity/v3/role_assignments?group.id=c1f75f60b9314d738b2ac7b030ea42d3&scope.project.id=523fc40cf4044e598d36fb05a352d260&role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839 => generated 260 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:09.535560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba5dce3-b20b-479f-9963-07ed14a63867 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:09.550661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:09.579417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb51bc-033e-426e-821b-00755a2cdb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:09.580206 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb51bc-033e-426e-821b-00755a2cdb69 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:09.580491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb51bc-033e-426e-821b-00755a2cdb69 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:09.580766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb51bc-033e-426e-821b-00755a2cdb69 None admin] PATH_INFO: `/v3/groups/8f17bdd7d46b48d48f54e619818b946b` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:09.586641 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb51bc-033e-426e-821b-00755a2cdb69 None admin] RBAC: Authorizing `identity:get_group(group_id=8f17bdd7d46b48d48f54e619818b946b)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:09.588185 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb51bc-033e-426e-821b-00755a2cdb69 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:09.590406 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 193/379] 10.222.0.39 () {56 vars in 1180 bytes} [Mon Aug 8 02:13:09 2022] GET /identity/v3/groups/8f17bdd7d46b48d48f54e619818b946b => generated 229 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:09.596093 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04090d-a392-48ec-85f6-4ef73940955a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:09.717820 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f723a84-3534-4ea1-91b7-edc7027defd0 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:09.718107 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f723a84-3534-4ea1-91b7-edc7027defd0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:09.718315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f723a84-3534-4ea1-91b7-edc7027defd0 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:09.719784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 194/380] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:13:09.729538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b7a2c-ffcb-4a00-8097-958831a264bd None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:09.729538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b7a2c-ffcb-4a00-8097-958831a264bd None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:09.729538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b7a2c-ffcb-4a00-8097-958831a264bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:09.745804 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04090d-a392-48ec-85f6-4ef73940955a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:09.752677 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-0e7b7a2c-ffcb-4a00-8097-958831a264bd None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:09.754592 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0e7b7a2c-ffcb-4a00-8097-958831a264bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:09.787312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787161df-40a6-4dfc-99db-e99300bca881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:09.788027 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787161df-40a6-4dfc-99db-e99300bca881 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:09.788305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787161df-40a6-4dfc-99db-e99300bca881 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:09.788567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787161df-40a6-4dfc-99db-e99300bca881 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:09.789393 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787161df-40a6-4dfc-99db-e99300bca881 None admin] RBAC: Adding query filter params (group.id=8f17bdd7d46b48d48f54e619818b946b, role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.project.id=520cd26c01a04fe5b710bc3899ff38a8) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:09.789702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787161df-40a6-4dfc-99db-e99300bca881 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:09.792383 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787161df-40a6-4dfc-99db-e99300bca881 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:09.798587 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 187/381] 10.222.0.39 () {56 vars in 1376 bytes} [Mon Aug 8 02:13:09 2022] GET /identity/v3/role_assignments?group.id=8f17bdd7d46b48d48f54e619818b946b&scope.project.id=520cd26c01a04fe5b710bc3899ff38a8&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:09.848385 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 195/382] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:09.873491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787161df-40a6-4dfc-99db-e99300bca881 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:09.939409 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:10.024277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787161df-40a6-4dfc-99db-e99300bca881 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:10.063044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766f086-34fa-4073-91de-bb9e4d0c2c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:10.063673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766f086-34fa-4073-91de-bb9e4d0c2c1d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:10.063949 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766f086-34fa-4073-91de-bb9e4d0c2c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:10.064212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766f086-34fa-4073-91de-bb9e4d0c2c1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:10.079576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766f086-34fa-4073-91de-bb9e4d0c2c1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:10.085486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7766f086-34fa-4073-91de-bb9e4d0c2c1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:10.085486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:10.085486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:10.085486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:10.085486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:10.085486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:10.085486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:10.085486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:10.085486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:10.085486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:10.085486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:10.085486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:10.085486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:10.085486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:10.085486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:10.085486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:10.086376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:10.086376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:10.086376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:10.086376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:10.086376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:10.086376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:10.086376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:10.086376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:10.086376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:10.086376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:10.086376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:10.086376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:10.086376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:10.086376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:10.086376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:10.086376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:10.087138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:10.087138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:10.087138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:10.087138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:10.087138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:10.087138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:10.087138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:10.087138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:10.087138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:10.087138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:10.087138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:10.087138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:10.087138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 188/383] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:13:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:10.095852 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b7a2c-ffcb-4a00-8097-958831a264bd None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:10.134063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e2d17-0b55-4c3b-b936-63b82da13254 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:10.134404 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e2d17-0b55-4c3b-b936-63b82da13254 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:10.134650 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e2d17-0b55-4c3b-b936-63b82da13254 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:10.136223 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 189/384] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:10.141143 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5a0bc-4399-4b91-8edc-ae0e1fa3238c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:10.141421 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5a0bc-4399-4b91-8edc-ae0e1fa3238c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:10.141693 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5a0bc-4399-4b91-8edc-ae0e1fa3238c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:10.170079 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c0e5a0bc-4399-4b91-8edc-ae0e1fa3238c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:10.171298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c0e5a0bc-4399-4b91-8edc-ae0e1fa3238c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:10.268037 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b7a2c-ffcb-4a00-8097-958831a264bd None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:10.272664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:10.277087 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 190/385] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:10.300101 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5a0bc-4399-4b91-8edc-ae0e1fa3238c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:10.343966 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:10.348055 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8ca68-aac8-4f5c-a244-ad8ff8d58da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:10.348672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8ca68-aac8-4f5c-a244-ad8ff8d58da6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:10.348978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8ca68-aac8-4f5c-a244-ad8ff8d58da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:10.349919 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8ca68-aac8-4f5c-a244-ad8ff8d58da6 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:10.351651 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8ca68-aac8-4f5c-a244-ad8ff8d58da6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:10.351954 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8ca68-aac8-4f5c-a244-ad8ff8d58da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:10.353346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8ca68-aac8-4f5c-a244-ad8ff8d58da6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:10.370191 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 196/386] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:10.376551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8ca68-aac8-4f5c-a244-ad8ff8d58da6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:10.449281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5a0bc-4399-4b91-8edc-ae0e1fa3238c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:10.511357 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41844b7-8b40-4019-8452-98f26a3f7368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:10.511990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41844b7-8b40-4019-8452-98f26a3f7368 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:10.512269 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41844b7-8b40-4019-8452-98f26a3f7368 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:10.512538 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41844b7-8b40-4019-8452-98f26a3f7368 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:10.522000 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8ca68-aac8-4f5c-a244-ad8ff8d58da6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:10.534866 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41844b7-8b40-4019-8452-98f26a3f7368 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:10.539654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b41844b7-8b40-4019-8452-98f26a3f7368 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:10.539654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:10.539654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:10.539654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:10.539654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:10.539654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:10.539654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:10.539654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:10.539654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:10.539654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:10.539654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:10.539654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:10.539654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:10.539654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:10.539654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:10.539654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:10.540528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:10.540528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:10.540528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:10.540528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:10.540528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:10.540528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:10.540528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:10.540528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:10.540528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:10.540528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:10.540528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:10.540528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:10.540528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:10.540528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:10.540528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:10.540528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:10.541800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:10.541800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:10.541800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:10.541800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:10.541800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:10.541800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:10.541800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:10.541800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:10.541800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:10.541800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:10.541800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:10.541800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:10.541800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 191/387] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:10.547803 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41844b7-8b40-4019-8452-98f26a3f7368 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:10.562491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a5e60-6ef9-404b-893f-def731a5238d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:10.563122 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a5e60-6ef9-404b-893f-def731a5238d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:10.563421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a5e60-6ef9-404b-893f-def731a5238d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:10.563763 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a5e60-6ef9-404b-893f-def731a5238d None admin] PATH_INFO: `/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:10.572507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a5e60-6ef9-404b-893f-def731a5238d None admin] RBAC: Authorizing `identity:get_group(group_id=c1f75f60b9314d738b2ac7b030ea42d3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:10.573969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a5e60-6ef9-404b-893f-def731a5238d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:10.579368 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 197/388] 10.222.0.39 () {56 vars in 1180 bytes} [Mon Aug 8 02:13:10 2022] GET /identity/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:10.582965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a5e60-6ef9-404b-893f-def731a5238d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:10.654008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:10.717908 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:10.755644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41844b7-8b40-4019-8452-98f26a3f7368 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:10.795142 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717697e-ddaf-46f7-a12a-be6fb8b824f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:10.795665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717697e-ddaf-46f7-a12a-be6fb8b824f9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:10.795944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717697e-ddaf-46f7-a12a-be6fb8b824f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:10.796217 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717697e-ddaf-46f7-a12a-be6fb8b824f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:10.797131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717697e-ddaf-46f7-a12a-be6fb8b824f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:10.797441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717697e-ddaf-46f7-a12a-be6fb8b824f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:10.799281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717697e-ddaf-46f7-a12a-be6fb8b824f9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:10.804583 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a5e60-6ef9-404b-893f-def731a5238d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:10.809353 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 192/389] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:10.815948 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717697e-ddaf-46f7-a12a-be6fb8b824f9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:10.865546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f7b91-18b9-4356-a361-5e5a4c89a939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:10.866199 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f7b91-18b9-4356-a361-5e5a4c89a939 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:10.867233 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f7b91-18b9-4356-a361-5e5a4c89a939 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:10.867514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f7b91-18b9-4356-a361-5e5a4c89a939 None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:10.868878 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f7b91-18b9-4356-a361-5e5a4c89a939 None admin] RBAC: Authorizing `identity:get_project(project_id=3441a04d36074741b13ebfa058e656b8)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:10.870340 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f7b91-18b9-4356-a361-5e5a4c89a939 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:10.873048 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 198/390] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:10 2022] GET /identity/v3/projects/3441a04d36074741b13ebfa058e656b8 => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:10.878896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f7b91-18b9-4356-a361-5e5a4c89a939 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:10.999429 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f7b91-18b9-4356-a361-5e5a4c89a939 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:11.000903 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:11.039025 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717697e-ddaf-46f7-a12a-be6fb8b824f9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:11.043811 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f26fe-b825-4d1d-9c9b-d3175b56ad4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:11.044672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f26fe-b825-4d1d-9c9b-d3175b56ad4d None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:11.045993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f26fe-b825-4d1d-9c9b-d3175b56ad4d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:11.046449 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f26fe-b825-4d1d-9c9b-d3175b56ad4d None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8/groups/c1f75f60b9314d738b2ac7b030ea42d3/roles/9f8a5b177f4a41fc8bc88ff4f1ce0839` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:11.055312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f26fe-b825-4d1d-9c9b-d3175b56ad4d None admin] RBAC: Authorizing `identity:create_grant(project_id=3441a04d36074741b13ebfa058e656b8, group_id=c1f75f60b9314d738b2ac7b030ea42d3, role_id=9f8a5b177f4a41fc8bc88ff4f1ce0839)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:11.057185 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f26fe-b825-4d1d-9c9b-d3175b56ad4d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:11.084455 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 199/391] 10.222.0.39 () {58 vars in 1442 bytes} [Mon Aug 8 02:13:10 2022] PUT /identity/v3/projects/3441a04d36074741b13ebfa058e656b8/groups/c1f75f60b9314d738b2ac7b030ea42d3/roles/9f8a5b177f4a41fc8bc88ff4f1ce0839 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:11.104258 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcb66d-f89f-4c57-8f8c-97289e94a477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:11.104918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcb66d-f89f-4c57-8f8c-97289e94a477 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:11.105192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcb66d-f89f-4c57-8f8c-97289e94a477 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:11.105457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcb66d-f89f-4c57-8f8c-97289e94a477 None admin] PATH_INFO: `/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:11.111179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fcb66d-f89f-4c57-8f8c-97289e94a477 None admin] RBAC: Authorizing `identity:get_group(group_id=c1f75f60b9314d738b2ac7b030ea42d3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:11.112701 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fcb66d-f89f-4c57-8f8c-97289e94a477 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:11.115441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 193/392] 10.222.0.39 () {56 vars in 1180 bytes} [Mon Aug 8 02:13:10 2022] GET /identity/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3 => generated 226 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:11.121089 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f26fe-b825-4d1d-9c9b-d3175b56ad4d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:11.272875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f26fe-b825-4d1d-9c9b-d3175b56ad4d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:11.315940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a211b6d-86c8-4f25-a579-a2de89fa9ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:11.316801 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a211b6d-86c8-4f25-a579-a2de89fa9ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:11.317243 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a211b6d-86c8-4f25-a579-a2de89fa9ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:11.317703 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a211b6d-86c8-4f25-a579-a2de89fa9ad3 None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:11.319067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a211b6d-86c8-4f25-a579-a2de89fa9ad3 None admin] RBAC: Authorizing `identity:get_project(project_id=3441a04d36074741b13ebfa058e656b8)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:11.320677 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a211b6d-86c8-4f25-a579-a2de89fa9ad3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:11.322873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 200/393] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:11 2022] GET /identity/v3/projects/3441a04d36074741b13ebfa058e656b8 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:11.328874 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcb66d-f89f-4c57-8f8c-97289e94a477 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:11.354207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:11.439960 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcb66d-f89f-4c57-8f8c-97289e94a477 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:11.474608 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84782216-d4d3-4700-9b6b-bdb522b1a947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:11.475596 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84782216-d4d3-4700-9b6b-bdb522b1a947 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:11.475875 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84782216-d4d3-4700-9b6b-bdb522b1a947 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:11.476140 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84782216-d4d3-4700-9b6b-bdb522b1a947 None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8/groups/c1f75f60b9314d738b2ac7b030ea42d3/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:11.478035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84782216-d4d3-4700-9b6b-bdb522b1a947 None admin] RBAC: Authorizing `identity:create_grant(project_id=3441a04d36074741b13ebfa058e656b8, group_id=c1f75f60b9314d738b2ac7b030ea42d3, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:11.479516 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84782216-d4d3-4700-9b6b-bdb522b1a947 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:11.507712 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 194/394] 10.222.0.39 () {58 vars in 1442 bytes} [Mon Aug 8 02:13:11 2022] PUT /identity/v3/projects/3441a04d36074741b13ebfa058e656b8/groups/c1f75f60b9314d738b2ac7b030ea42d3/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:11.664532 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7698d42-d5e0-4d03-922b-ee38392afbad None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:11.664772 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7698d42-d5e0-4d03-922b-ee38392afbad None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:11.665151 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7698d42-d5e0-4d03-922b-ee38392afbad None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:11.666932 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 201/395] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:11.672747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258e903-4dc5-42fe-b2da-613f686d9d2e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:11.673039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258e903-4dc5-42fe-b2da-613f686d9d2e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:11.673311 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258e903-4dc5-42fe-b2da-613f686d9d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:11.699390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b258e903-4dc5-42fe-b2da-613f686d9d2e None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:11.700698 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b258e903-4dc5-42fe-b2da-613f686d9d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:11.789795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 195/396] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:11.819756 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7698d42-d5e0-4d03-922b-ee38392afbad None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:11.864765 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:11.960770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7698d42-d5e0-4d03-922b-ee38392afbad None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:12.002319 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52406cee-da80-476d-bf68-99e76a3f6d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:12.002943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52406cee-da80-476d-bf68-99e76a3f6d07 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:12.003216 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52406cee-da80-476d-bf68-99e76a3f6d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:12.003484 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52406cee-da80-476d-bf68-99e76a3f6d07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:12.018787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52406cee-da80-476d-bf68-99e76a3f6d07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:12.021778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9a398-92e8-4b58-954e-8145a4a2fb35 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:12.022069 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9a398-92e8-4b58-954e-8145a4a2fb35 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:12.022388 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9a398-92e8-4b58-954e-8145a4a2fb35 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:12.023979 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 196/397] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:12.024432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-52406cee-da80-476d-bf68-99e76a3f6d07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:12.024432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:12.024432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:12.024432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:12.024432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:12.024432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:12.024432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:12.024432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:12.024432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:12.024432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:12.024432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:12.024432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:12.024432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:12.024432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:12.024432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:12.024432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:12.025472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:12.025472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:12.025472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:12.025472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:12.025472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:12.025472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:12.025472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:12.025472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:12.025472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:12.025472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:12.025472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:12.025472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:12.025472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:12.025472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:12.025472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:12.025472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:12.026525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:12.026525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:12.026525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:12.026525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:12.026525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:12.026525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:12.026525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:12.026525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:12.026525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:12.026525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:12.026525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:12.026525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:12.026525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 202/398] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:12.028760 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e0eb6-bdbe-4c8d-b6fe-0957a70eb530 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:12.029044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e0eb6-bdbe-4c8d-b6fe-0957a70eb530 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:12.029318 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e0eb6-bdbe-4c8d-b6fe-0957a70eb530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:12.033639 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52406cee-da80-476d-bf68-99e76a3f6d07 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:12.044583 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:12.063505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-980e0eb6-bdbe-4c8d-b6fe-0957a70eb530 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:12.064794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-980e0eb6-bdbe-4c8d-b6fe-0957a70eb530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:12.188690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:12.205402 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 197/399] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:12.207158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52406cee-da80-476d-bf68-99e76a3f6d07 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:12.227262 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e0eb6-bdbe-4c8d-b6fe-0957a70eb530 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:12.262040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36861b-c846-4a47-87da-3b0972684875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:12.262926 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36861b-c846-4a47-87da-3b0972684875 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:12.263344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36861b-c846-4a47-87da-3b0972684875 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:12.263768 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36861b-c846-4a47-87da-3b0972684875 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:12.264563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36861b-c846-4a47-87da-3b0972684875 None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:12.265765 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36861b-c846-4a47-87da-3b0972684875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:12.270020 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36861b-c846-4a47-87da-3b0972684875 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:12.288550 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 203/400] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:12.291363 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36861b-c846-4a47-87da-3b0972684875 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:12.381196 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e0eb6-bdbe-4c8d-b6fe-0957a70eb530 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:12.408685 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36861b-c846-4a47-87da-3b0972684875 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:12.415904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8c455-5c9d-4d7b-a113-77ed99d0db15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:12.416205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8c455-5c9d-4d7b-a113-77ed99d0db15 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:12.416499 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8c455-5c9d-4d7b-a113-77ed99d0db15 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:12.416758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8c455-5c9d-4d7b-a113-77ed99d0db15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:12.426741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:12.428745 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8c455-5c9d-4d7b-a113-77ed99d0db15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:12.433446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-cdc8c455-5c9d-4d7b-a113-77ed99d0db15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:12.433446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:12.433446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:12.433446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:12.433446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:12.433446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:12.433446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:12.433446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:12.433446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:12.433446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:12.433446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:12.433446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:12.433446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:12.433446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:12.433446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:12.433446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:12.435089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:12.435089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:12.435089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:12.435089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:12.435089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:12.435089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:12.435089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:12.435089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:12.435089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:12.435089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:12.435089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:12.435089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:12.435089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:12.435089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:12.435089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:12.435089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:12.436220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:12.436220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:12.436220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:12.436220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:12.436220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:12.436220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:12.436220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:12.436220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:12.436220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:12.436220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:12.436220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:12.436220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:12.436220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 198/401] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:13:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:12.437867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa5764-c56e-4465-9b50-8f7131d2f543 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:12.438153 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa5764-c56e-4465-9b50-8f7131d2f543 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:12.438466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa5764-c56e-4465-9b50-8f7131d2f543 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:12.439993 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 199/402] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:12.443567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa5764-c56e-4465-9b50-8f7131d2f543 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:12.462162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a007f20-a4ca-4ebe-8b46-695dfb0a4888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:12.463159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a007f20-a4ca-4ebe-8b46-695dfb0a4888 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:12.463632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a007f20-a4ca-4ebe-8b46-695dfb0a4888 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:12.464253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a007f20-a4ca-4ebe-8b46-695dfb0a4888 None admin] PATH_INFO: `/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:12.465582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a007f20-a4ca-4ebe-8b46-695dfb0a4888 None admin] RBAC: Authorizing `identity:get_group(group_id=c1f75f60b9314d738b2ac7b030ea42d3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:12.467344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a007f20-a4ca-4ebe-8b46-695dfb0a4888 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:12.473613 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 204/403] 10.222.0.39 () {56 vars in 1180 bytes} [Mon Aug 8 02:13:12 2022] GET /identity/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:12.475128 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a647af-b85c-4e6d-86d8-7dc1521ab519 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:12.475623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a647af-b85c-4e6d-86d8-7dc1521ab519 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:12.476088 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a647af-b85c-4e6d-86d8-7dc1521ab519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:12.494942 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-31a647af-b85c-4e6d-86d8-7dc1521ab519 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:12.496470 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-31a647af-b85c-4e6d-86d8-7dc1521ab519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:12.524479 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:12.553437 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa5764-c56e-4465-9b50-8f7131d2f543 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:12.568245 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 205/404] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:12.569435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a647af-b85c-4e6d-86d8-7dc1521ab519 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:12.590911 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b23c1e-7714-424f-9587-346894fd493f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:12.591724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b23c1e-7714-424f-9587-346894fd493f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:12.592151 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b23c1e-7714-424f-9587-346894fd493f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:12.592569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b23c1e-7714-424f-9587-346894fd493f None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:12.593387 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b23c1e-7714-424f-9587-346894fd493f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:12.593842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b23c1e-7714-424f-9587-346894fd493f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:12.595571 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b23c1e-7714-424f-9587-346894fd493f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:12.605649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 200/405] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:12.606934 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b23c1e-7714-424f-9587-346894fd493f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:12.672807 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:12.675121 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a647af-b85c-4e6d-86d8-7dc1521ab519 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:12.711037 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f416a-e965-48d3-a9fb-92e30f6e254c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:12.711836 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f416a-e965-48d3-a9fb-92e30f6e254c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:12.712235 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f416a-e965-48d3-a9fb-92e30f6e254c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:12.712545 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f416a-e965-48d3-a9fb-92e30f6e254c None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:12.713765 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f416a-e965-48d3-a9fb-92e30f6e254c None admin] RBAC: Authorizing `identity:get_project(project_id=523fc40cf4044e598d36fb05a352d260)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:12.715321 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f416a-e965-48d3-a9fb-92e30f6e254c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:12.721720 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b23c1e-7714-424f-9587-346894fd493f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:12.722044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 206/406] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:12 2022] GET /identity/v3/projects/523fc40cf4044e598d36fb05a352d260 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:12.722884 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f416a-e965-48d3-a9fb-92e30f6e254c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:12.743585 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:12.763126 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd884bed-0868-452c-b904-b4597476eac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:12.764054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd884bed-0868-452c-b904-b4597476eac8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:12.764537 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd884bed-0868-452c-b904-b4597476eac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:12.765033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd884bed-0868-452c-b904-b4597476eac8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:12.776392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd884bed-0868-452c-b904-b4597476eac8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:12.782018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-cd884bed-0868-452c-b904-b4597476eac8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:12.782018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:12.782018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:12.782018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:12.782018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:12.782018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:12.782018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:12.782018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:12.782018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:12.782018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:12.782018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:12.782018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:12.782018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:12.782018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:12.782018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:12.782018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:12.782914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:12.782914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:12.782914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:12.782914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:12.782914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:12.782914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:12.782914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:12.782914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:12.782914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:12.782914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:12.782914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:12.782914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:12.782914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:12.782914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:12.782914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:12.782914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:12.783677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:12.783677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:12.783677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:12.783677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:12.783677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:12.783677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:12.783677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:12.783677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:12.783677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:12.783677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:12.783677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:12.783677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:12.786999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 201/407] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:12.787986 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd884bed-0868-452c-b904-b4597476eac8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:12.828938 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f416a-e965-48d3-a9fb-92e30f6e254c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:12.864422 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddeaad6-fc37-489f-b3ce-3d239a1bc85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:12.865563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddeaad6-fc37-489f-b3ce-3d239a1bc85e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:12.865836 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddeaad6-fc37-489f-b3ce-3d239a1bc85e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:12.866104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddeaad6-fc37-489f-b3ce-3d239a1bc85e None admin] PATH_INFO: `/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:12.867251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddeaad6-fc37-489f-b3ce-3d239a1bc85e None admin] RBAC: Authorizing `identity:get_group(group_id=c1f75f60b9314d738b2ac7b030ea42d3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:12.868739 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddeaad6-fc37-489f-b3ce-3d239a1bc85e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:12.870564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 207/408] 10.222.0.39 () {56 vars in 1180 bytes} [Mon Aug 8 02:13:12 2022] GET /identity/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:12.871291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddeaad6-fc37-489f-b3ce-3d239a1bc85e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:12.903780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd884bed-0868-452c-b904-b4597476eac8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:12.923295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:12.944214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f47e3-b428-49d9-944f-0cee7413541d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:12.945303 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f47e3-b428-49d9-944f-0cee7413541d None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:12.945881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f47e3-b428-49d9-944f-0cee7413541d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:12.946415 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f47e3-b428-49d9-944f-0cee7413541d None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260/groups/c1f75f60b9314d738b2ac7b030ea42d3/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:12.948599 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f47e3-b428-49d9-944f-0cee7413541d None admin] RBAC: Authorizing `identity:create_grant(project_id=523fc40cf4044e598d36fb05a352d260, group_id=c1f75f60b9314d738b2ac7b030ea42d3, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:12.950531 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f47e3-b428-49d9-944f-0cee7413541d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:12.965758 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:12.966629 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 202/409] 10.222.0.39 () {58 vars in 1442 bytes} [Mon Aug 8 02:13:12 2022] PUT /identity/v3/projects/523fc40cf4044e598d36fb05a352d260/groups/c1f75f60b9314d738b2ac7b030ea42d3/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:12.967800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f47e3-b428-49d9-944f-0cee7413541d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:12.996519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddeaad6-fc37-489f-b3ce-3d239a1bc85e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:13.067505 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc039d-4eb2-441c-ac80-cc1075056349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:13.068192 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc039d-4eb2-441c-ac80-cc1075056349 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:13.068542 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc039d-4eb2-441c-ac80-cc1075056349 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:13.068808 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc039d-4eb2-441c-ac80-cc1075056349 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:13.069482 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc039d-4eb2-441c-ac80-cc1075056349 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:13.069783 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc039d-4eb2-441c-ac80-cc1075056349 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:13.071293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc039d-4eb2-441c-ac80-cc1075056349 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:13.090787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 208/410] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:13.093957 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f47e3-b428-49d9-944f-0cee7413541d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:13.096463 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc039d-4eb2-441c-ac80-cc1075056349 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:13.133899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a755ea5-7c57-419d-a0c4-44773f6c203d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:13.134978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a755ea5-7c57-419d-a0c4-44773f6c203d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:13.135492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a755ea5-7c57-419d-a0c4-44773f6c203d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:13.136024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a755ea5-7c57-419d-a0c4-44773f6c203d None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:13.137668 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a755ea5-7c57-419d-a0c4-44773f6c203d None admin] RBAC: Authorizing `identity:get_project(project_id=523fc40cf4044e598d36fb05a352d260)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:13.139543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a755ea5-7c57-419d-a0c4-44773f6c203d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:13.141640 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 203/411] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:12 2022] GET /identity/v3/projects/523fc40cf4044e598d36fb05a352d260 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:13.147893 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a755ea5-7c57-419d-a0c4-44773f6c203d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:13.216506 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc039d-4eb2-441c-ac80-cc1075056349 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:13.236144 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:13.254978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c134e-4a0c-4e49-979d-79932c25b314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:13.255638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c134e-4a0c-4e49-979d-79932c25b314 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:13.255917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c134e-4a0c-4e49-979d-79932c25b314 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:13.256190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c134e-4a0c-4e49-979d-79932c25b314 None admin] PATH_INFO: `/v3/groups/8f17bdd7d46b48d48f54e619818b946b` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:13.257299 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c134e-4a0c-4e49-979d-79932c25b314 None admin] RBAC: Authorizing `identity:get_group(group_id=8f17bdd7d46b48d48f54e619818b946b)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:13.258752 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c134e-4a0c-4e49-979d-79932c25b314 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:13.260666 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 209/412] 10.222.0.39 () {56 vars in 1180 bytes} [Mon Aug 8 02:13:13 2022] GET /identity/v3/groups/8f17bdd7d46b48d48f54e619818b946b => generated 229 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:13.267730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c134e-4a0c-4e49-979d-79932c25b314 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:13.292014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:13.310638 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a755ea5-7c57-419d-a0c4-44773f6c203d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:13.375340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b75895-2efd-475e-97bd-75621ea8b3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:13.376460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b75895-2efd-475e-97bd-75621ea8b3de None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:13.377094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b75895-2efd-475e-97bd-75621ea8b3de None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:13.377581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b75895-2efd-475e-97bd-75621ea8b3de None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260/groups/c1f75f60b9314d738b2ac7b030ea42d3/roles/9f8a5b177f4a41fc8bc88ff4f1ce0839` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:13.379822 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c134e-4a0c-4e49-979d-79932c25b314 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:13.380843 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b75895-2efd-475e-97bd-75621ea8b3de None admin] RBAC: Authorizing `identity:create_grant(project_id=523fc40cf4044e598d36fb05a352d260, group_id=c1f75f60b9314d738b2ac7b030ea42d3, role_id=9f8a5b177f4a41fc8bc88ff4f1ce0839)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:13.382728 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b75895-2efd-475e-97bd-75621ea8b3de None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:13.418460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 204/413] 10.222.0.39 () {58 vars in 1442 bytes} [Mon Aug 8 02:13:13 2022] PUT /identity/v3/projects/523fc40cf4044e598d36fb05a352d260/groups/c1f75f60b9314d738b2ac7b030ea42d3/roles/9f8a5b177f4a41fc8bc88ff4f1ce0839 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:13.424023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731f80f-f088-4762-bb65-5f3327daa30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:13.424823 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731f80f-f088-4762-bb65-5f3327daa30a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:13.425106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731f80f-f088-4762-bb65-5f3327daa30a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:13.425379 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731f80f-f088-4762-bb65-5f3327daa30a None admin] PATH_INFO: `/v3/projects/520cd26c01a04fe5b710bc3899ff38a8` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:13.426802 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731f80f-f088-4762-bb65-5f3327daa30a None admin] RBAC: Authorizing `identity:get_project(project_id=520cd26c01a04fe5b710bc3899ff38a8)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:13.428477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731f80f-f088-4762-bb65-5f3327daa30a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:13.430782 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 210/414] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:13 2022] GET /identity/v3/projects/520cd26c01a04fe5b710bc3899ff38a8 => generated 344 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:13.436318 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b75895-2efd-475e-97bd-75621ea8b3de None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:13.586831 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee1ceb-c8b0-44b7-abd9-8dbeccf5693b None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:13.587116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee1ceb-c8b0-44b7-abd9-8dbeccf5693b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:13.587382 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee1ceb-c8b0-44b7-abd9-8dbeccf5693b None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:13.588866 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 211/415] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:13.596927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ba5ce-0d49-4b69-b1dd-e49dbc16f594 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:13.596927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ba5ce-0d49-4b69-b1dd-e49dbc16f594 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:13.596927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ba5ce-0d49-4b69-b1dd-e49dbc16f594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:13.606455 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b75895-2efd-475e-97bd-75621ea8b3de None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:13.614584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:13.615234 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-123ba5ce-0d49-4b69-b1dd-e49dbc16f594 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:13.616661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-123ba5ce-0d49-4b69-b1dd-e49dbc16f594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:13.652602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b07e17-e3a8-4afa-ada4-cc56173d4bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:13.653194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b07e17-e3a8-4afa-ada4-cc56173d4bc3 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:13.653468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b07e17-e3a8-4afa-ada4-cc56173d4bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:13.653739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b07e17-e3a8-4afa-ada4-cc56173d4bc3 None admin] PATH_INFO: `/v3/projects/520cd26c01a04fe5b710bc3899ff38a8/groups/8f17bdd7d46b48d48f54e619818b946b/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:13.656483 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b07e17-e3a8-4afa-ada4-cc56173d4bc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=520cd26c01a04fe5b710bc3899ff38a8, group_id=8f17bdd7d46b48d48f54e619818b946b, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:13.658241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b07e17-e3a8-4afa-ada4-cc56173d4bc3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:13.679597 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 205/416] 10.222.0.39 () {58 vars in 1442 bytes} [Mon Aug 8 02:13:13 2022] PUT /identity/v3/projects/520cd26c01a04fe5b710bc3899ff38a8/groups/8f17bdd7d46b48d48f54e619818b946b/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:13.705776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 212/417] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:13.730343 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b07e17-e3a8-4afa-ada4-cc56173d4bc3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:13.866994 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b07e17-e3a8-4afa-ada4-cc56173d4bc3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:13.904936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390c4b2-96f3-4f1c-aa78-7184a0b55228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:13.905932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390c4b2-96f3-4f1c-aa78-7184a0b55228 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:13.906448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390c4b2-96f3-4f1c-aa78-7184a0b55228 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:13.906928 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390c4b2-96f3-4f1c-aa78-7184a0b55228 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:13.918486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0390c4b2-96f3-4f1c-aa78-7184a0b55228 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:13.924275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0390c4b2-96f3-4f1c-aa78-7184a0b55228 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:13.924275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:13.924275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:13.924275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:13.924275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:13.924275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:13.924275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:13.924275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:13.924275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:13.924275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:13.924275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:13.924275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:13.924275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:13.924275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:13.924275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:13.924275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:13.925209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:13.925209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:13.925209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:13.925209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:13.925209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:13.925209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:13.925209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:13.925209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:13.925209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:13.925209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:13.925209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:13.925209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:13.925209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:13.925209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:13.925209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:13.925209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:13.926024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:13.926024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:13.926024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:13.926024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:13.926024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:13.926024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:13.926024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:13.926024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:13.926024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:13.926024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:13.926024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:13.926024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:13.927746 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234a58c-96a4-4df8-bf52-54a009fb92ec None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:13.928032 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234a58c-96a4-4df8-bf52-54a009fb92ec None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:13.928303 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234a58c-96a4-4df8-bf52-54a009fb92ec None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:13.929498 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 206/418] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:13:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:13.934269 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 213/419] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:13.935991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234a58c-96a4-4df8-bf52-54a009fb92ec None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:13.936262 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113061a9-83bd-41e3-98cf-2b676ace27b3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:13.936807 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113061a9-83bd-41e3-98cf-2b676ace27b3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:13.937273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113061a9-83bd-41e3-98cf-2b676ace27b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:13.956677 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:13.959110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-113061a9-83bd-41e3-98cf-2b676ace27b3 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:13.960323 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-113061a9-83bd-41e3-98cf-2b676ace27b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:13.985421 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:14.043236 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 207/420] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:14.066239 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113061a9-83bd-41e3-98cf-2b676ace27b3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:14.072361 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234a58c-96a4-4df8-bf52-54a009fb92ec None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:14.109069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c48e0f-5035-464b-ad4f-7bfca783a0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:14.109749 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c48e0f-5035-464b-ad4f-7bfca783a0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:14.110079 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c48e0f-5035-464b-ad4f-7bfca783a0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:14.110390 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c48e0f-5035-464b-ad4f-7bfca783a0a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:14.111097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c48e0f-5035-464b-ad4f-7bfca783a0a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:14.111800 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c48e0f-5035-464b-ad4f-7bfca783a0a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:14.112908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c48e0f-5035-464b-ad4f-7bfca783a0a2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:14.122015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 214/421] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:14.127458 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c48e0f-5035-464b-ad4f-7bfca783a0a2 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:14.175664 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113061a9-83bd-41e3-98cf-2b676ace27b3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:14.212895 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dda418-a561-4736-abba-a922eae6bec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:14.214355 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dda418-a561-4736-abba-a922eae6bec3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:14.214838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dda418-a561-4736-abba-a922eae6bec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:14.215317 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dda418-a561-4736-abba-a922eae6bec3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:14.228358 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dda418-a561-4736-abba-a922eae6bec3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:14.233410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a0dda418-a561-4736-abba-a922eae6bec3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:14.233410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:14.233410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:14.233410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:14.233410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:14.233410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:14.233410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:14.233410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:14.233410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:14.233410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:14.233410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:14.233410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:14.233410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:14.233410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:14.233410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:14.233410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:14.234305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:14.234305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:14.234305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:14.234305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:14.234305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:14.234305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:14.234305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:14.234305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:14.234305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:14.234305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:14.234305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:14.234305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:14.234305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:14.234305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:14.234305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:14.234305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:14.235100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:14.235100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:14.235100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:14.235100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:14.235100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:14.235100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:14.235100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:14.235100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:14.235100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:14.235100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:14.235100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:14.235100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:14.237306 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 208/422] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:14.243341 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dda418-a561-4736-abba-a922eae6bec3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:14.264854 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:14.286638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:14.340409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c48e0f-5035-464b-ad4f-7bfca783a0a2 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:14.353955 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dda418-a561-4736-abba-a922eae6bec3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:14.389732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62a4f8-f75f-446e-8321-455145b861b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:14.391168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62a4f8-f75f-446e-8321-455145b861b2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:14.391653 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62a4f8-f75f-446e-8321-455145b861b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:14.392117 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62a4f8-f75f-446e-8321-455145b861b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:14.392955 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62a4f8-f75f-446e-8321-455145b861b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:14.393445 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62a4f8-f75f-446e-8321-455145b861b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:14.395099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62a4f8-f75f-446e-8321-455145b861b2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:14.404492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 209/423] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:14.410133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62a4f8-f75f-446e-8321-455145b861b2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:14.446778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189db685-9421-45ac-b551-e14a445cb8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:14.447319 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189db685-9421-45ac-b551-e14a445cb8bc None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:14.447599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189db685-9421-45ac-b551-e14a445cb8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:14.448232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189db685-9421-45ac-b551-e14a445cb8bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:14.521570 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189db685-9421-45ac-b551-e14a445cb8bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:14.524137 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189db685-9421-45ac-b551-e14a445cb8bc None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:14.571995 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:14.609652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:14.640655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 215/424] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 02:13:14.644422 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62a4f8-f75f-446e-8321-455145b861b2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:14.646352 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189db685-9421-45ac-b551-e14a445cb8bc None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:14.706066 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c2754-12a2-4243-a22f-a659939399bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:14.706648 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c2754-12a2-4243-a22f-a659939399bc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:14.706928 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c2754-12a2-4243-a22f-a659939399bc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:14.707196 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c2754-12a2-4243-a22f-a659939399bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:14.837359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189db685-9421-45ac-b551-e14a445cb8bc None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:14.852276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c2754-12a2-4243-a22f-a659939399bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:14.853962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c2754-12a2-4243-a22f-a659939399bc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:14.878084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1012cc-b78b-4f4c-aabe-238cf627e07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:14.879478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1012cc-b78b-4f4c-aabe-238cf627e07f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:14.879954 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1012cc-b78b-4f4c-aabe-238cf627e07f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:14.880459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1012cc-b78b-4f4c-aabe-238cf627e07f None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:14.881979 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1012cc-b78b-4f4c-aabe-238cf627e07f None admin] RBAC: Authorizing `identity:get_project(project_id=3441a04d36074741b13ebfa058e656b8)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:14.883881 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1012cc-b78b-4f4c-aabe-238cf627e07f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:14.885992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 216/425] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:14 2022] GET /identity/v3/projects/3441a04d36074741b13ebfa058e656b8 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:14.890864 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1012cc-b78b-4f4c-aabe-238cf627e07f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:14.950723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:14.991145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:15.011209 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1012cc-b78b-4f4c-aabe-238cf627e07f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:15.013400 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 210/426] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:15.027224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c2754-12a2-4243-a22f-a659939399bc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:15.051718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026f888-fa6f-4a92-9e15-1cf93fecf2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:15.053348 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026f888-fa6f-4a92-9e15-1cf93fecf2fd None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:15.053884 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026f888-fa6f-4a92-9e15-1cf93fecf2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:15.054439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026f888-fa6f-4a92-9e15-1cf93fecf2fd None admin] PATH_INFO: `/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:15.055800 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026f888-fa6f-4a92-9e15-1cf93fecf2fd None admin] RBAC: Authorizing `identity:get_group(group_id=c1f75f60b9314d738b2ac7b030ea42d3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:15.057415 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026f888-fa6f-4a92-9e15-1cf93fecf2fd None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:15.059494 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 217/427] 10.222.0.39 () {56 vars in 1180 bytes} [Mon Aug 8 02:13:14 2022] GET /identity/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:15.065796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026f888-fa6f-4a92-9e15-1cf93fecf2fd None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:15.168199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c2754-12a2-4243-a22f-a659939399bc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:15.239498 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d768df-ab51-472c-b95e-2b9e29dcfd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:15.241216 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d768df-ab51-472c-b95e-2b9e29dcfd8b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:15.243251 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d768df-ab51-472c-b95e-2b9e29dcfd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:15.243722 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d768df-ab51-472c-b95e-2b9e29dcfd8b None admin] PATH_INFO: `/v3/projects/3441a04d36074741b13ebfa058e656b8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:15.245128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d768df-ab51-472c-b95e-2b9e29dcfd8b None admin] RBAC: Authorizing `identity:get_project(project_id=3441a04d36074741b13ebfa058e656b8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:15.247748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d768df-ab51-472c-b95e-2b9e29dcfd8b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:15.252895 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 211/428] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:15 2022] GET /identity/v3/projects/3441a04d36074741b13ebfa058e656b8 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:15.276547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026f888-fa6f-4a92-9e15-1cf93fecf2fd None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:15.279934 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d768df-ab51-472c-b95e-2b9e29dcfd8b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:15.310676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:15.357494 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118f609-2bcc-43c2-bd3c-d02624b9e143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:15.358121 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118f609-2bcc-43c2-bd3c-d02624b9e143 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:15.358430 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118f609-2bcc-43c2-bd3c-d02624b9e143 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:15.358710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118f609-2bcc-43c2-bd3c-d02624b9e143 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:15.359463 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118f609-2bcc-43c2-bd3c-d02624b9e143 None admin] RBAC: Adding query filter params (group.id=c1f75f60b9314d738b2ac7b030ea42d3, role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839, scope.project.id=3441a04d36074741b13ebfa058e656b8) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:15.359764 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118f609-2bcc-43c2-bd3c-d02624b9e143 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:15.361188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118f609-2bcc-43c2-bd3c-d02624b9e143 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:15.361594 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:15.375551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 218/429] 10.222.0.39 () {56 vars in 1376 bytes} [Mon Aug 8 02:13:15 2022] GET /identity/v3/role_assignments?group.id=c1f75f60b9314d738b2ac7b030ea42d3&scope.project.id=3441a04d36074741b13ebfa058e656b8&role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839 => generated 611 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:15.447237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d768df-ab51-472c-b95e-2b9e29dcfd8b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:15.507391 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a844d-007a-4ad4-b538-273d77f6acf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:15.508047 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a844d-007a-4ad4-b538-273d77f6acf6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:15.508327 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a844d-007a-4ad4-b538-273d77f6acf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:15.508603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a844d-007a-4ad4-b538-273d77f6acf6 None admin] PATH_INFO: `/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:15.509765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a844d-007a-4ad4-b538-273d77f6acf6 None admin] RBAC: Authorizing `identity:get_group(group_id=c1f75f60b9314d738b2ac7b030ea42d3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:15.511258 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a844d-007a-4ad4-b538-273d77f6acf6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:15.513202 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 212/430] 10.222.0.39 () {56 vars in 1180 bytes} [Mon Aug 8 02:13:15 2022] GET /identity/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:15.518750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118f609-2bcc-43c2-bd3c-d02624b9e143 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:15.627683 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118f609-2bcc-43c2-bd3c-d02624b9e143 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:15.664367 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662f876-51b9-492a-9c43-84418100d4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:15.665450 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662f876-51b9-492a-9c43-84418100d4cf None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:15.665977 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662f876-51b9-492a-9c43-84418100d4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:15.666487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662f876-51b9-492a-9c43-84418100d4cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:15.667484 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662f876-51b9-492a-9c43-84418100d4cf None admin] RBAC: Adding query filter params (group.id=c1f75f60b9314d738b2ac7b030ea42d3, role.id=42c846f6944b40c8af5eeb8360cb7c7a, scope.project.id=3441a04d36074741b13ebfa058e656b8) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:15.667991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662f876-51b9-492a-9c43-84418100d4cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:15.669617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662f876-51b9-492a-9c43-84418100d4cf None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:15.676144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 219/431] 10.222.0.39 () {56 vars in 1376 bytes} [Mon Aug 8 02:13:15 2022] GET /identity/v3/role_assignments?group.id=c1f75f60b9314d738b2ac7b030ea42d3&scope.project.id=3441a04d36074741b13ebfa058e656b8&role.id=42c846f6944b40c8af5eeb8360cb7c7a => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:15.739259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251a4f6-768d-4671-a188-9522534bc207 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:15.739983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251a4f6-768d-4671-a188-9522534bc207 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:15.740457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251a4f6-768d-4671-a188-9522534bc207 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:15.742116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 213/432] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:15.748114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c201b-3dc2-4d2a-8251-ecf2c02dd950 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:15.748708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c201b-3dc2-4d2a-8251-ecf2c02dd950 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:15.749199 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c201b-3dc2-4d2a-8251-ecf2c02dd950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:15.769982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-3a0c201b-3dc2-4d2a-8251-ecf2c02dd950 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:15.772139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3a0c201b-3dc2-4d2a-8251-ecf2c02dd950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:15.806724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4164a-876f-4370-9776-64898b92611b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:15.807072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4164a-876f-4370-9776-64898b92611b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:15.807257 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4164a-876f-4370-9776-64898b92611b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:15.808947 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 214/433] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:15.812101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:15.815906 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3621f6-d091-426e-8e70-5504b0f1291f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:15.818325 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3621f6-d091-426e-8e70-5504b0f1291f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:15.818325 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3621f6-d091-426e-8e70-5504b0f1291f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:15.842551 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5f3621f6-d091-426e-8e70-5504b0f1291f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:15.843669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5f3621f6-d091-426e-8e70-5504b0f1291f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:15.851888 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 220/434] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:15.876689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c201b-3dc2-4d2a-8251-ecf2c02dd950 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:15.942228 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 215/435] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:15.965753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3621f6-d091-426e-8e70-5504b0f1291f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:15.992447 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c201b-3dc2-4d2a-8251-ecf2c02dd950 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:16.017696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:16.033520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf8453-3eef-40a8-95b0-b00444c61ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:16.034074 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf8453-3eef-40a8-95b0-b00444c61ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:16.034384 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf8453-3eef-40a8-95b0-b00444c61ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:16.034655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf8453-3eef-40a8-95b0-b00444c61ad6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:16.051273 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf8453-3eef-40a8-95b0-b00444c61ad6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:16.057664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-afaf8453-3eef-40a8-95b0-b00444c61ad6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:16.057664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:16.057664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:16.057664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:16.057664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:16.057664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:16.057664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:16.057664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:16.057664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:16.057664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:16.057664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:16.057664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:16.057664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:16.057664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:16.057664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:16.057664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:16.058657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:16.058657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:16.058657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:16.058657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:16.058657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:16.058657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:16.058657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:16.058657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:16.058657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:16.058657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:16.058657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:16.058657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:16.058657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:16.058657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:16.058657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:16.058657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:16.059440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:16.059440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:16.059440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:16.059440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:16.059440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:16.059440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:16.059440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:16.059440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:16.059440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:16.059440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:16.059440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:16.059440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:16.060333 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 221/436] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:16.072684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf8453-3eef-40a8-95b0-b00444c61ad6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:16.099501 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3621f6-d091-426e-8e70-5504b0f1291f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:16.115683 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:16.149085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa5236-adac-4a21-96b6-251715aa6b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:16.150022 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa5236-adac-4a21-96b6-251715aa6b6c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:16.153332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa5236-adac-4a21-96b6-251715aa6b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:16.153332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa5236-adac-4a21-96b6-251715aa6b6c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:16.165288 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa5236-adac-4a21-96b6-251715aa6b6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:16.171005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f1fa5236-adac-4a21-96b6-251715aa6b6c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:16.171005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:16.171005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:16.171005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:16.171005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:16.171005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:16.171005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:16.171005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:16.171005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:16.171005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:16.171005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:16.171005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:16.171005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:16.171005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:16.171005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:16.171005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:16.171863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:16.171863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:16.171863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:16.171863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:16.171863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:16.171863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:16.171863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:16.171863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:16.171863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:16.171863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:16.171863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:16.171863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:16.171863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:16.171863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:16.171863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:16.171863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:16.172624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:16.172624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:16.172624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:16.172624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:16.172624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:16.172624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:16.172624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:16.172624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:16.172624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:16.172624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:16.172624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:16.172624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:16.173209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 216/437] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:13:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:16.182869 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa5236-adac-4a21-96b6-251715aa6b6c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:16.207362 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf8453-3eef-40a8-95b0-b00444c61ad6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:16.260148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f892a66-e387-4993-9777-408f569bc019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:16.260588 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f892a66-e387-4993-9777-408f569bc019 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:16.260944 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f892a66-e387-4993-9777-408f569bc019 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:16.261285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f892a66-e387-4993-9777-408f569bc019 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:16.262044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f892a66-e387-4993-9777-408f569bc019 None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:16.263184 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f892a66-e387-4993-9777-408f569bc019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:16.266278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f892a66-e387-4993-9777-408f569bc019 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:16.274897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 222/438] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:16.280387 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f892a66-e387-4993-9777-408f569bc019 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:16.314337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa5236-adac-4a21-96b6-251715aa6b6c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:16.317863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:16.354215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2abba1-8c38-4865-91db-f531789f77f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:16.354844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2abba1-8c38-4865-91db-f531789f77f7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:16.355117 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2abba1-8c38-4865-91db-f531789f77f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:16.355382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2abba1-8c38-4865-91db-f531789f77f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:16.356041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2abba1-8c38-4865-91db-f531789f77f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:16.356341 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2abba1-8c38-4865-91db-f531789f77f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:16.358223 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2abba1-8c38-4865-91db-f531789f77f7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:16.366532 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 217/439] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:16.367707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93f0c3-3131-4f4d-9d05-8424faca7a1f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:16.367992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93f0c3-3131-4f4d-9d05-8424faca7a1f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:16.368255 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93f0c3-3131-4f4d-9d05-8424faca7a1f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:16.369421 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 218/440] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:16.371719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93f0c3-3131-4f4d-9d05-8424faca7a1f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:16.389351 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:16.398693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f892a66-e387-4993-9777-408f569bc019 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:16.434306 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b188e-455d-4e49-a740-e6a8d202388e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:16.435185 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b188e-455d-4e49-a740-e6a8d202388e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:16.435463 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b188e-455d-4e49-a740-e6a8d202388e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:16.435737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b188e-455d-4e49-a740-e6a8d202388e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:16.476305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93f0c3-3131-4f4d-9d05-8424faca7a1f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:16.501997 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b188e-455d-4e49-a740-e6a8d202388e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:16.503478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b188e-455d-4e49-a740-e6a8d202388e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:16.508593 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6319e9-5865-4b00-9c5b-cca1ca930836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:16.509186 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6319e9-5865-4b00-9c5b-cca1ca930836 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:16.509463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6319e9-5865-4b00-9c5b-cca1ca930836 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:16.509734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6319e9-5865-4b00-9c5b-cca1ca930836 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:16.541907 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:16.576508 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6319e9-5865-4b00-9c5b-cca1ca930836 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:16.577973 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6319e9-5865-4b00-9c5b-cca1ca930836 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:16.593275 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:16.603585 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 223/441] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 02:13:16.604814 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c43886-3e1d-4b8c-ae03-89851d600f86 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:16.605084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c43886-3e1d-4b8c-ae03-89851d600f86 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:16.605354 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c43886-3e1d-4b8c-ae03-89851d600f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:16.622624 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d1c43886-3e1d-4b8c-ae03-89851d600f86 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:16.623824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d1c43886-3e1d-4b8c-ae03-89851d600f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:16.674947 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 219/442] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 02:13:16.675853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6319e9-5865-4b00-9c5b-cca1ca930836 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:16.690230 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 224/443] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:16.691076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c43886-3e1d-4b8c-ae03-89851d600f86 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:16.752593 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:16.789076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c43886-3e1d-4b8c-ae03-89851d600f86 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:16.789266 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6319e9-5865-4b00-9c5b-cca1ca930836 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:16.811858 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:16.823434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f9f9d-915e-480d-b653-e50cf7c12920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:16.823580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9afc49-a0e4-4dab-9c4e-92820b5e44ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:16.824056 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f9f9d-915e-480d-b653-e50cf7c12920 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:16.824180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9afc49-a0e4-4dab-9c4e-92820b5e44ee None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:16.824331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f9f9d-915e-480d-b653-e50cf7c12920 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:16.824460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9afc49-a0e4-4dab-9c4e-92820b5e44ee None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:16.824599 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f9f9d-915e-480d-b653-e50cf7c12920 None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:16.824738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9afc49-a0e4-4dab-9c4e-92820b5e44ee None admin] PATH_INFO: `/v3/projects/523fc40cf4044e598d36fb05a352d260` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:16.825709 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f9f9d-915e-480d-b653-e50cf7c12920 None admin] RBAC: Authorizing `identity:get_project(project_id=523fc40cf4044e598d36fb05a352d260)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:16.825842 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9afc49-a0e4-4dab-9c4e-92820b5e44ee None admin] RBAC: Authorizing `identity:get_project(project_id=523fc40cf4044e598d36fb05a352d260)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:16.827239 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f9f9d-915e-480d-b653-e50cf7c12920 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:16.827345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9afc49-a0e4-4dab-9c4e-92820b5e44ee None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:16.833912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 220/444] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:16 2022] GET /identity/v3/projects/523fc40cf4044e598d36fb05a352d260 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:16.834048 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 225/445] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:16 2022] GET /identity/v3/projects/523fc40cf4044e598d36fb05a352d260 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:16.834160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f9f9d-915e-480d-b653-e50cf7c12920 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:16.834636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9afc49-a0e4-4dab-9c4e-92820b5e44ee None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:16.929301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f9f9d-915e-480d-b653-e50cf7c12920 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:16.930789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9afc49-a0e4-4dab-9c4e-92820b5e44ee None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:16.963416 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb308b-d60c-438e-a50e-c5075f2ecb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:16.964031 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb308b-d60c-438e-a50e-c5075f2ecb12 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:16.964310 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb308b-d60c-438e-a50e-c5075f2ecb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:16.964583 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb308b-d60c-438e-a50e-c5075f2ecb12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:16.964672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72b1d3-2e9d-4144-8a96-03c89663e7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:16.965197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72b1d3-2e9d-4144-8a96-03c89663e7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:16.965472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72b1d3-2e9d-4144-8a96-03c89663e7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:16.965738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72b1d3-2e9d-4144-8a96-03c89663e7c3 None admin] PATH_INFO: `/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:16.966449 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:16.967015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b72b1d3-2e9d-4144-8a96-03c89663e7c3 None admin] RBAC: Authorizing `identity:get_group(group_id=c1f75f60b9314d738b2ac7b030ea42d3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:16.968431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b72b1d3-2e9d-4144-8a96-03c89663e7c3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:16.970165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 226/446] 10.222.0.39 () {56 vars in 1180 bytes} [Mon Aug 8 02:13:16 2022] GET /identity/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3 => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:16.975141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72b1d3-2e9d-4144-8a96-03c89663e7c3 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:16.976516 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb308b-d60c-438e-a50e-c5075f2ecb12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:16.981191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-99cb308b-d60c-438e-a50e-c5075f2ecb12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:16.981191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:16.981191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:16.981191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:16.981191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:16.981191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:16.981191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:16.981191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:16.981191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:16.981191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:16.981191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:16.981191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:16.981191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:16.981191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:16.981191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:16.981191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:16.982062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:16.982062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:16.982062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:16.982062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:16.982062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:16.982062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:16.982062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:16.982062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:16.982062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:16.982062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:16.982062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:16.982062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:16.982062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:16.982062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:16.982062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:16.982062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:16.982851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:16.982851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:16.982851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:16.982851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:16.982851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:16.982851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:16.982851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:16.982851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:16.982851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:16.982851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:16.982851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:16.982851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:16.982851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 221/447] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:16.983915 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb308b-d60c-438e-a50e-c5075f2ecb12 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:17.035529 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:17.077483 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72b1d3-2e9d-4144-8a96-03c89663e7c3 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:17.081147 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb308b-d60c-438e-a50e-c5075f2ecb12 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:17.110522 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d94302-eab2-47c2-8e19-9c996f6d1389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:17.111134 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d94302-eab2-47c2-8e19-9c996f6d1389 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:17.111413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d94302-eab2-47c2-8e19-9c996f6d1389 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:17.111679 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d94302-eab2-47c2-8e19-9c996f6d1389 None admin] PATH_INFO: `/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:17.112751 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d94302-eab2-47c2-8e19-9c996f6d1389 None admin] RBAC: Authorizing `identity:get_group(group_id=c1f75f60b9314d738b2ac7b030ea42d3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:17.114163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d94302-eab2-47c2-8e19-9c996f6d1389 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:17.114601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a2f0d-b497-4673-a94e-562d1935a809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:17.115195 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a2f0d-b497-4673-a94e-562d1935a809 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:17.115469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a2f0d-b497-4673-a94e-562d1935a809 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:17.115732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a2f0d-b497-4673-a94e-562d1935a809 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:17.115839 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 227/448] 10.222.0.39 () {56 vars in 1180 bytes} [Mon Aug 8 02:13:16 2022] GET /identity/v3/groups/c1f75f60b9314d738b2ac7b030ea42d3 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:17.116442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a2f0d-b497-4673-a94e-562d1935a809 None admin] RBAC: Adding query filter params (group.id=c1f75f60b9314d738b2ac7b030ea42d3, role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839, scope.project.id=523fc40cf4044e598d36fb05a352d260) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:17.116537 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d94302-eab2-47c2-8e19-9c996f6d1389 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:17.116748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a2f0d-b497-4673-a94e-562d1935a809 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:17.118162 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a2f0d-b497-4673-a94e-562d1935a809 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:17.138281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 222/449] 10.222.0.39 () {56 vars in 1376 bytes} [Mon Aug 8 02:13:16 2022] GET /identity/v3/role_assignments?group.id=c1f75f60b9314d738b2ac7b030ea42d3&scope.project.id=523fc40cf4044e598d36fb05a352d260&role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:17.138281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a2f0d-b497-4673-a94e-562d1935a809 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:17.188761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:17.242275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a2f0d-b497-4673-a94e-562d1935a809 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:17.243721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d94302-eab2-47c2-8e19-9c996f6d1389 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:17.279784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:17.296716 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f45d2-50e6-4f4c-a058-e2ad97044d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:17.296716 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f45d2-50e6-4f4c-a058-e2ad97044d46 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:17.296716 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f45d2-50e6-4f4c-a058-e2ad97044d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:17.296716 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f45d2-50e6-4f4c-a058-e2ad97044d46 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:17.297627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f45d2-50e6-4f4c-a058-e2ad97044d46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:17.298666 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f45d2-50e6-4f4c-a058-e2ad97044d46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:17.301174 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f45d2-50e6-4f4c-a058-e2ad97044d46 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:17.314342 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 228/450] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:17.317336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa28e59-5b4f-43e6-8c54-af4db1f9cee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:17.318717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa28e59-5b4f-43e6-8c54-af4db1f9cee5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:17.319128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa28e59-5b4f-43e6-8c54-af4db1f9cee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:17.319529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa28e59-5b4f-43e6-8c54-af4db1f9cee5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:17.320920 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa28e59-5b4f-43e6-8c54-af4db1f9cee5 None admin] RBAC: Adding query filter params (group.id=c1f75f60b9314d738b2ac7b030ea42d3, role.id=42c846f6944b40c8af5eeb8360cb7c7a, scope.project.id=523fc40cf4044e598d36fb05a352d260) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:17.321031 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa28e59-5b4f-43e6-8c54-af4db1f9cee5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:17.321367 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f45d2-50e6-4f4c-a058-e2ad97044d46 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:17.323185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa28e59-5b4f-43e6-8c54-af4db1f9cee5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:17.334513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 223/451] 10.222.0.39 () {56 vars in 1376 bytes} [Mon Aug 8 02:13:17 2022] GET /identity/v3/role_assignments?group.id=c1f75f60b9314d738b2ac7b030ea42d3&scope.project.id=523fc40cf4044e598d36fb05a352d260&role.id=42c846f6944b40c8af5eeb8360cb7c7a => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:17.450725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f45d2-50e6-4f4c-a058-e2ad97044d46 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:17.487681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fdc636-4559-4ee6-ae2f-414f91ba67b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:17.488293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fdc636-4559-4ee6-ae2f-414f91ba67b4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:17.488566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fdc636-4559-4ee6-ae2f-414f91ba67b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:17.488834 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fdc636-4559-4ee6-ae2f-414f91ba67b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:17.556485 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:17.572909 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fdc636-4559-4ee6-ae2f-414f91ba67b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:17.574419 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fdc636-4559-4ee6-ae2f-414f91ba67b4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:17.677509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 229/452] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:17.683260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa28e59-5b4f-43e6-8c54-af4db1f9cee5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:17.785075 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa28e59-5b4f-43e6-8c54-af4db1f9cee5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:17.807134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:17.822226 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7bed6-c72c-4f18-95dc-a85826a3ccf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:17.822875 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7bed6-c72c-4f18-95dc-a85826a3ccf7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:17.823152 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7bed6-c72c-4f18-95dc-a85826a3ccf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:17.823422 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7bed6-c72c-4f18-95dc-a85826a3ccf7 None admin] PATH_INFO: `/v3/projects/520cd26c01a04fe5b710bc3899ff38a8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:17.824544 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7bed6-c72c-4f18-95dc-a85826a3ccf7 None admin] RBAC: Authorizing `identity:get_project(project_id=520cd26c01a04fe5b710bc3899ff38a8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:17.827462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7bed6-c72c-4f18-95dc-a85826a3ccf7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:17.827873 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 224/453] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:17 2022] GET /identity/v3/projects/520cd26c01a04fe5b710bc3899ff38a8 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:17.833768 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fdc636-4559-4ee6-ae2f-414f91ba67b4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:17.916882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:17.929223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fdc636-4559-4ee6-ae2f-414f91ba67b4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:17.962839 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9c3c9-6e11-469d-ab1c-cd89ca949e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:17.963464 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9c3c9-6e11-469d-ab1c-cd89ca949e40 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:17.963746 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9c3c9-6e11-469d-ab1c-cd89ca949e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:17.964018 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9c3c9-6e11-469d-ab1c-cd89ca949e40 None admin] PATH_INFO: `/v3/groups/8f17bdd7d46b48d48f54e619818b946b` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:17.965142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9c3c9-6e11-469d-ab1c-cd89ca949e40 None admin] RBAC: Authorizing `identity:get_group(group_id=8f17bdd7d46b48d48f54e619818b946b)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:17.966588 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9c3c9-6e11-469d-ab1c-cd89ca949e40 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:17.968159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 230/454] 10.222.0.39 () {56 vars in 1180 bytes} [Mon Aug 8 02:13:17 2022] GET /identity/v3/groups/8f17bdd7d46b48d48f54e619818b946b => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:17.975185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7bed6-c72c-4f18-95dc-a85826a3ccf7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:18.073607 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7bed6-c72c-4f18-95dc-a85826a3ccf7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:18.105595 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ed17f-a8c4-4d12-8c47-f4e6654f7b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:18.106172 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ed17f-a8c4-4d12-8c47-f4e6654f7b3b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:18.106482 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ed17f-a8c4-4d12-8c47-f4e6654f7b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:18.106750 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ed17f-a8c4-4d12-8c47-f4e6654f7b3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:18.107453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ed17f-a8c4-4d12-8c47-f4e6654f7b3b None admin] RBAC: Adding query filter params (group.id=8f17bdd7d46b48d48f54e619818b946b, role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.project.id=520cd26c01a04fe5b710bc3899ff38a8) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:18.107755 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ed17f-a8c4-4d12-8c47-f4e6654f7b3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:18.109174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ed17f-a8c4-4d12-8c47-f4e6654f7b3b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:18.114724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 225/455] 10.222.0.39 () {56 vars in 1376 bytes} [Mon Aug 8 02:13:17 2022] GET /identity/v3/role_assignments?group.id=8f17bdd7d46b48d48f54e619818b946b&scope.project.id=520cd26c01a04fe5b710bc3899ff38a8&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:20.503680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252547ec-74c1-40be-a34f-091692a48a39 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:20.504606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252547ec-74c1-40be-a34f-091692a48a39 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:20.505157 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252547ec-74c1-40be-a34f-091692a48a39 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:20.506998 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 231/456] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:20.512433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4679eb1-d41e-4638-87af-9357f50ed2e9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:20.513204 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4679eb1-d41e-4638-87af-9357f50ed2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:20.513622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4679eb1-d41e-4638-87af-9357f50ed2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:20.543199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c4679eb1-d41e-4638-87af-9357f50ed2e9 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:20.544626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c4679eb1-d41e-4638-87af-9357f50ed2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:20.613932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:20.639752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 226/457] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:20.683020 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252547ec-74c1-40be-a34f-091692a48a39 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:20.807965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252547ec-74c1-40be-a34f-091692a48a39 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:20.855147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da972293-fb09-4534-870f-0b4482999ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:20.855781 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da972293-fb09-4534-870f-0b4482999ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:20.856111 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da972293-fb09-4534-870f-0b4482999ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:20.856382 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da972293-fb09-4534-870f-0b4482999ef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:20.869723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da972293-fb09-4534-870f-0b4482999ef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:20.871196 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da972293-fb09-4534-870f-0b4482999ef3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:20.893991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-da972293-fb09-4534-870f-0b4482999ef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:20.893991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:20.893991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:20.893991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:20.893991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:20.893991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:20.893991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:20.893991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:20.893991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:20.893991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:20.893991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:20.893991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:20.893991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:20.893991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:20.893991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:20.893991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:20.894976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:20.894976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:20.894976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:20.894976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:20.894976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:20.894976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:20.894976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:20.894976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:20.894976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:20.894976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:20.894976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:20.894976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:20.894976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:20.894976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:20.894976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:20.894976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:20.894976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:20.896394 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:20.896394 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:20.896394 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:20.898631 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 232/458] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:20.907356 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4679eb1-d41e-4638-87af-9357f50ed2e9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:20.962094 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77962b8d-dc4d-4000-ab46-6ab6c0224d4a None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:20.962744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77962b8d-dc4d-4000-ab46-6ab6c0224d4a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:20.963219 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77962b8d-dc4d-4000-ab46-6ab6c0224d4a None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:20.964859 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 233/459] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:13:20.974845 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae52cd8-08c2-4b00-9d03-6580840acfae None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:20.975121 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae52cd8-08c2-4b00-9d03-6580840acfae None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:20.975390 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae52cd8-08c2-4b00-9d03-6580840acfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:20.999184 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-5ae52cd8-08c2-4b00-9d03-6580840acfae None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:21.000378 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5ae52cd8-08c2-4b00-9d03-6580840acfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:21.002680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:21.025079 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4679eb1-d41e-4638-87af-9357f50ed2e9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:21.060029 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0caf87b-5419-4301-aa66-c7ec85ba12a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:21.060546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0caf87b-5419-4301-aa66-c7ec85ba12a7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:21.060821 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0caf87b-5419-4301-aa66-c7ec85ba12a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:21.061088 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0caf87b-5419-4301-aa66-c7ec85ba12a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:21.061727 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0caf87b-5419-4301-aa66-c7ec85ba12a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:21.062029 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0caf87b-5419-4301-aa66-c7ec85ba12a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:21.063432 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0caf87b-5419-4301-aa66-c7ec85ba12a7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:21.077383 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 227/460] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:21.078847 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256b769-d040-40c1-89c0-ec767a5c8485 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:21.079302 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256b769-d040-40c1-89c0-ec767a5c8485 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:21.081846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256b769-d040-40c1-89c0-ec767a5c8485 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:21.081846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 228/461] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:21.082707 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 234/462] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:21.086555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256b769-d040-40c1-89c0-ec767a5c8485 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:21.087442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3950b-25e5-4f17-88ae-60441a92a2d9 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:21.087839 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3950b-25e5-4f17-88ae-60441a92a2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:21.088202 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3950b-25e5-4f17-88ae-60441a92a2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:21.107390 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-16e3950b-25e5-4f17-88ae-60441a92a2d9 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:21.108710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-16e3950b-25e5-4f17-88ae-60441a92a2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:21.134340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:21.185013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 235/463] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:21.185883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3950b-25e5-4f17-88ae-60441a92a2d9 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:21.208570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256b769-d040-40c1-89c0-ec767a5c8485 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:21.248921 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28f734-910b-47f4-9aff-c985c5d57b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:21.249820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28f734-910b-47f4-9aff-c985c5d57b74 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:21.250163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28f734-910b-47f4-9aff-c985c5d57b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:21.250529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28f734-910b-47f4-9aff-c985c5d57b74 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:21.251228 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28f734-910b-47f4-9aff-c985c5d57b74 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:21.252932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28f734-910b-47f4-9aff-c985c5d57b74 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:21.271549 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:21.279005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 229/464] 10.222.0.39 () {60 vars in 1132 bytes} [Mon Aug 8 02:13:21 2022] POST /identity/v3/users => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:21.280016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28f734-910b-47f4-9aff-c985c5d57b74 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:21.311158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3950b-25e5-4f17-88ae-60441a92a2d9 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:21.350917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfe682-3c93-4290-bc23-aa4c2b332bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:21.351706 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfe682-3c93-4290-bc23-aa4c2b332bbe None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:21.352055 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfe682-3c93-4290-bc23-aa4c2b332bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:21.352400 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfe682-3c93-4290-bc23-aa4c2b332bbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:21.364822 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bfe682-3c93-4290-bc23-aa4c2b332bbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:21.366680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bfe682-3c93-4290-bc23-aa4c2b332bbe None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:21.384471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-f0bfe682-3c93-4290-bc23-aa4c2b332bbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:21.384471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:21.384471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:21.384471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:21.384471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:21.384471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:21.384471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:21.384471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:21.384471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:21.384471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:21.384471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:21.384471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:21.384471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:21.384471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:21.384471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:21.384471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:21.386571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:21.386571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:21.386571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:21.386571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:21.386571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:21.386571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:21.386571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:21.386571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:21.386571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:21.386571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:21.386571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:21.386571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:21.386571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:21.386571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:21.386571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:21.386571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:21.386571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:21.387674 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:21.387674 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:21.387674 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:21.387674 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 236/465] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:21.393826 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfe682-3c93-4290-bc23-aa4c2b332bbe None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:21.395774 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:21.401894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28f734-910b-47f4-9aff-c985c5d57b74 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:21.477286 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d61b3-75d9-4fb3-b9a8-16d928d40134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:21.481769 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d61b3-75d9-4fb3-b9a8-16d928d40134 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:21.482037 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d61b3-75d9-4fb3-b9a8-16d928d40134 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:21.482329 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d61b3-75d9-4fb3-b9a8-16d928d40134 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:21.504091 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d61b3-75d9-4fb3-b9a8-16d928d40134 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:21.506321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d61b3-75d9-4fb3-b9a8-16d928d40134 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:21.522576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7f2d61b3-75d9-4fb3-b9a8-16d928d40134 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:21.522576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:21.522576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:21.522576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:21.522576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:21.522576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:21.522576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:21.522576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:21.522576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:21.522576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:21.522576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:21.522576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:21.522576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:21.522576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:21.522576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:21.522576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:21.524005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:21.524005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:21.524005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:21.524005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:21.524005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:21.524005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:21.524005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:21.524005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:21.524005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:21.524005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:21.524005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:21.524005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:21.524005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:21.524005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:21.524005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:21.524005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:21.524005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:21.525030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:21.525030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:21.525030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:21.525030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 230/466] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:21.525030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48050b2b-5c82-47f3-8f5e-5d19b7c0d278 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:21.525354 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48050b2b-5c82-47f3-8f5e-5d19b7c0d278 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:21.525354 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48050b2b-5c82-47f3-8f5e-5d19b7c0d278 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:21.530286 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 231/467] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:21.531360 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48050b2b-5c82-47f3-8f5e-5d19b7c0d278 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:21.577788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfe682-3c93-4290-bc23-aa4c2b332bbe None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:21.601524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:21.618480 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f189c-0061-4158-bd78-0aaaa843e72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:21.619383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f189c-0061-4158-bd78-0aaaa843e72a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:21.619832 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f189c-0061-4158-bd78-0aaaa843e72a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:21.620272 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f189c-0061-4158-bd78-0aaaa843e72a None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:21.621134 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f189c-0061-4158-bd78-0aaaa843e72a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:21.621611 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f189c-0061-4158-bd78-0aaaa843e72a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:21.623169 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f189c-0061-4158-bd78-0aaaa843e72a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:21.637651 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 237/468] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:21.638972 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979518e-46a2-4539-b557-cb39b6949dbe None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:21.639265 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979518e-46a2-4539-b557-cb39b6949dbe None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:21.639540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979518e-46a2-4539-b557-cb39b6949dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:21.660699 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d979518e-46a2-4539-b557-cb39b6949dbe None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:21.662467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d979518e-46a2-4539-b557-cb39b6949dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:21.663179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48050b2b-5c82-47f3-8f5e-5d19b7c0d278 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:21.704516 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abeca39-b2b2-4909-ba99-6b50ec903a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:21.706367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abeca39-b2b2-4909-ba99-6b50ec903a9d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:21.706850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abeca39-b2b2-4909-ba99-6b50ec903a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:21.707313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abeca39-b2b2-4909-ba99-6b50ec903a9d None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:21.708194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abeca39-b2b2-4909-ba99-6b50ec903a9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:21.708717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abeca39-b2b2-4909-ba99-6b50ec903a9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:21.710722 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abeca39-b2b2-4909-ba99-6b50ec903a9d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:21.728886 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 232/469] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:21.729976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abeca39-b2b2-4909-ba99-6b50ec903a9d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:21.744485 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 238/470] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:21.745607 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac5af5-60b2-41be-94d6-7e9eb3b2e44a None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:21.745876 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac5af5-60b2-41be-94d6-7e9eb3b2e44a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:21.746159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac5af5-60b2-41be-94d6-7e9eb3b2e44a None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:21.747391 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 239/471] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:21.750166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:21.752131 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac5af5-60b2-41be-94d6-7e9eb3b2e44a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:21.847461 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abeca39-b2b2-4909-ba99-6b50ec903a9d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:21.860096 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac5af5-60b2-41be-94d6-7e9eb3b2e44a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:21.871231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:21.886906 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f11c4-ac42-4843-b169-7f0f7a975004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:21.887486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f11c4-ac42-4843-b169-7f0f7a975004 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:21.887763 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f11c4-ac42-4843-b169-7f0f7a975004 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:21.888059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f11c4-ac42-4843-b169-7f0f7a975004 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:21.888692 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f11c4-ac42-4843-b169-7f0f7a975004 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:21.890147 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f11c4-ac42-4843-b169-7f0f7a975004 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:21.898036 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8390b-c36b-4728-b6bf-a3fc9d004f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:21.899022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8390b-c36b-4728-b6bf-a3fc9d004f6f None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:21.899867 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8390b-c36b-4728-b6bf-a3fc9d004f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:21.900636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8390b-c36b-4728-b6bf-a3fc9d004f6f None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:21.901736 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8390b-c36b-4728-b6bf-a3fc9d004f6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:21.903636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8390b-c36b-4728-b6bf-a3fc9d004f6f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:21.913080 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 233/472] 10.222.0.39 () {60 vars in 1132 bytes} [Mon Aug 8 02:13:21 2022] POST /identity/v3/users => generated 248 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:21.914477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795778b-7e06-4aca-8312-aa8cc084adf1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:21.914752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795778b-7e06-4aca-8312-aa8cc084adf1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:21.915020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795778b-7e06-4aca-8312-aa8cc084adf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:21.935298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7795778b-7e06-4aca-8312-aa8cc084adf1 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:21.936528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7795778b-7e06-4aca-8312-aa8cc084adf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:21.948340 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 240/473] 10.222.0.39 () {60 vars in 1132 bytes} [Mon Aug 8 02:13:21 2022] POST /identity/v3/users => generated 248 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:21.950038 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8390b-c36b-4728-b6bf-a3fc9d004f6f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:22.066650 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:22.078456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 234/474] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:22.103476 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795778b-7e06-4aca-8312-aa8cc084adf1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:22.150682 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8390b-c36b-4728-b6bf-a3fc9d004f6f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:22.196302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633055a9-3a27-4f76-ac3a-15216e3e8771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:22.197239 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633055a9-3a27-4f76-ac3a-15216e3e8771 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:22.197239 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633055a9-3a27-4f76-ac3a-15216e3e8771 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:22.197506 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633055a9-3a27-4f76-ac3a-15216e3e8771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:22.210965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633055a9-3a27-4f76-ac3a-15216e3e8771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:22.212605 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633055a9-3a27-4f76-ac3a-15216e3e8771 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:22.231194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-633055a9-3a27-4f76-ac3a-15216e3e8771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:22.231194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:22.231194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:22.231194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:22.231194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:22.231194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:22.231194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:22.231194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:22.231194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:22.231194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:22.231194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:22.231194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:22.231194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:22.231194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:22.231194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:22.231194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:22.232148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:22.232148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:22.232148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:22.232148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:22.232148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:22.232148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:22.232148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:22.232148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:22.232148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:22.232148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:22.232148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:22.232148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:22.232148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:22.232148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:22.232148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:22.232148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:22.232148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:22.233377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:22.233377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:22.233377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:22.233377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 241/475] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:22.233826 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795778b-7e06-4aca-8312-aa8cc084adf1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:22.238974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633055a9-3a27-4f76-ac3a-15216e3e8771 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:22.261475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:22.276945 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acb817-160b-4ccf-b07d-89dfdc39d530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:22.278112 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acb817-160b-4ccf-b07d-89dfdc39d530 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:22.278426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acb817-160b-4ccf-b07d-89dfdc39d530 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:22.278696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acb817-160b-4ccf-b07d-89dfdc39d530 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:22.279366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acb817-160b-4ccf-b07d-89dfdc39d530 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:22.280884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acb817-160b-4ccf-b07d-89dfdc39d530 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:22.296837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.sql.core [None req-c6acb817-160b-4ccf-b07d-89dfdc39d530 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 02:13:22.296837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 02:13:22.296837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [parameters: {'id': 'c8ce34174225438a8624c20cd018a40d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 02:13:22.296837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80160) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 02:13:22.313716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c6acb817-160b-4ccf-b07d-89dfdc39d530 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 02:13:22.313716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:22.313716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 02:13:22.313716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 02:13:22.313716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 02:13:22.313716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 02:13:22.313716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 02:13:22.313716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 02:13:22.313716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 02:13:22.313716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application conn.query(q) Aug 08 02:13:22.313716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 02:13:22.313716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 02:13:22.313716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 02:13:22.313716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result.read() Aug 08 02:13:22.313716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 02:13:22.314677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 02:13:22.314677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 02:13:22.314677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 02:13:22.314677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 02:13:22.314677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 02:13:22.314677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 02:13:22.314677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 02:13:22.314677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 02:13:22.314677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:22.314677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 02:13:22.314677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:22.314677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:22.314677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 02:13:22.314677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 02:13:22.314677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 08 02:13:22.314677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return ref.to_dict() Aug 08 02:13:22.314677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 02:13:22.314677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application next(self.gen) Aug 08 02:13:22.315870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 02:13:22.315870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application yield resource Aug 08 02:13:22.315870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 02:13:22.315870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application next(self.gen) Aug 08 02:13:22.315870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 02:13:22.315870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 02:13:22.315870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 02:13:22.315870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 02:13:22.315870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 02:13:22.315870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise self.value Aug 08 02:13:22.315870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 02:13:22.315870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 02:13:22.315870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 02:13:22.315870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application session.commit() Aug 08 02:13:22.315870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 02:13:22.315870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 02:13:22.315870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 02:13:22.317016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 02:13:22.317016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 02:13:22.317016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.session.flush() Aug 08 02:13:22.317016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 02:13:22.317016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 02:13:22.317016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 02:13:22.317016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 02:13:22.317016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 02:13:22.317016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application compat.raise_( Aug 08 02:13:22.317016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 02:13:22.317016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception Aug 08 02:13:22.317016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 02:13:22.317016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 02:13:22.317016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 02:13:22.317016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 02:13:22.317016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 02:13:22.317016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 02:13:22.318611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 02:13:22.318611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 02:13:22.318611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 02:13:22.318611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 02:13:22.318611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 02:13:22.318611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 02:13:22.318611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 02:13:22.318611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 02:13:22.318611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 02:13:22.318611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 02:13:22.318611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 02:13:22.318611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 02:13:22.318611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 02:13:22.318611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 02:13:22.318611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 02:13:22.318611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception Aug 08 02:13:22.319601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 02:13:22.319601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 02:13:22.319601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 02:13:22.319601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 02:13:22.319601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 02:13:22.319601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 02:13:22.319601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 02:13:22.319601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application conn.query(q) Aug 08 02:13:22.319601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 02:13:22.319601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 02:13:22.319601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 02:13:22.319601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result.read() Aug 08 02:13:22.319601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 02:13:22.319601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 02:13:22.319601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 02:13:22.319601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 02:13:22.320578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 02:13:22.320578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 02:13:22.320578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 02:13:22.320578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 02:13:22.320578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 02:13:22.320578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 02:13:22.320578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [parameters: {'id': 'c8ce34174225438a8624c20cd018a40d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 02:13:22.320578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 02:13:22.320578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:22.320578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:13:22.320578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:22.320578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:22.320578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:22.320578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:22.320578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:22.320578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:22.321602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:22.321602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:22.321602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:22.321602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:22.321602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:22.321602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:22.321602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 08 02:13:22.321602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 08 02:13:22.321602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:22.321602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:22.321602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 08 02:13:22.321602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 08 02:13:22.321602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 02:13:22.321602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 02:13:22.321602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 02:13:22.321602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:22.322710 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 235/476] 10.222.0.39 () {60 vars in 1131 bytes} [Mon Aug 8 02:13:22 2022] POST /identity/v3/roles => generated 144 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:22.324314 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acb817-160b-4ccf-b07d-89dfdc39d530 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:22.378344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633055a9-3a27-4f76-ac3a-15216e3e8771 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:22.393585 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:22.425024 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fd503-e91b-43d1-965d-a9785f2aca16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:22.425948 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fd503-e91b-43d1-965d-a9785f2aca16 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:22.426430 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fd503-e91b-43d1-965d-a9785f2aca16 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:22.427411 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fd503-e91b-43d1-965d-a9785f2aca16 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:22.428249 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2fd503-e91b-43d1-965d-a9785f2aca16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:22.428738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2fd503-e91b-43d1-965d-a9785f2aca16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:22.430474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2fd503-e91b-43d1-965d-a9785f2aca16 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:22.439364 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acb817-160b-4ccf-b07d-89dfdc39d530 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:22.444709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 242/477] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:22.452819 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fd503-e91b-43d1-965d-a9785f2aca16 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:22.484177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7010da-5ec6-492f-854c-2874a9f51613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:22.484177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7010da-5ec6-492f-854c-2874a9f51613 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:22.484177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7010da-5ec6-492f-854c-2874a9f51613 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:22.484177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7010da-5ec6-492f-854c-2874a9f51613 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:22.494313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7010da-5ec6-492f-854c-2874a9f51613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:22.499533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-2e7010da-5ec6-492f-854c-2874a9f51613 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:22.499533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:22.499533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:22.499533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:22.499533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:22.499533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:22.499533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:22.499533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:22.499533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:22.499533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:22.499533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:22.499533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:22.499533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:22.499533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:22.499533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:22.499533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:22.500375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:22.500375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:22.500375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:22.500375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:22.500375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:22.500375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:22.500375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:22.500375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:22.500375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:22.500375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:22.500375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:22.500375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:22.500375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:22.500375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:22.500375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:22.500375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:22.501588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:22.501588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:22.501588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:22.501588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:22.501588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:22.501588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:22.501588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:22.501588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:22.501588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:22.501588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:22.501588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:22.501588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:22.501588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 236/478] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:13:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:22.506210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7010da-5ec6-492f-854c-2874a9f51613 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:22.565714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:22.578566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fd503-e91b-43d1-965d-a9785f2aca16 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:22.617268 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60a1af-3f9e-4af0-96f0-6ab01e03555d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:22.618204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60a1af-3f9e-4af0-96f0-6ab01e03555d None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:22.618880 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60a1af-3f9e-4af0-96f0-6ab01e03555d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:22.619334 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60a1af-3f9e-4af0-96f0-6ab01e03555d None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:22.620287 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60a1af-3f9e-4af0-96f0-6ab01e03555d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:22.622023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60a1af-3f9e-4af0-96f0-6ab01e03555d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:22.662025 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 243/479] 10.222.0.39 () {60 vars in 1132 bytes} [Mon Aug 8 02:13:22 2022] POST /identity/v3/users => generated 249 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:22.695067 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7010da-5ec6-492f-854c-2874a9f51613 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:22.734479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:22.756637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65f01b-d0a5-4286-9914-e84cb38f17e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:22.757262 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65f01b-d0a5-4286-9914-e84cb38f17e1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:22.757539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65f01b-d0a5-4286-9914-e84cb38f17e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:22.757808 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65f01b-d0a5-4286-9914-e84cb38f17e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:22.758550 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65f01b-d0a5-4286-9914-e84cb38f17e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:22.758857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65f01b-d0a5-4286-9914-e84cb38f17e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:22.760290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65f01b-d0a5-4286-9914-e84cb38f17e1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:22.771016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 237/480] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:23.812584 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243c741-b4be-40bf-8ede-8cd558abfafd None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:23.813099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243c741-b4be-40bf-8ede-8cd558abfafd None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:23.813204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243c741-b4be-40bf-8ede-8cd558abfafd None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:23.814810 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 244/481] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:23.822498 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532f46d-6501-425e-bdbf-9e0ad925568c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:23.822921 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532f46d-6501-425e-bdbf-9e0ad925568c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:23.823091 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532f46d-6501-425e-bdbf-9e0ad925568c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:23.847334 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7532f46d-6501-425e-bdbf-9e0ad925568c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:23.848579 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7532f46d-6501-425e-bdbf-9e0ad925568c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:23.938165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 238/482] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 08 02:13:23.973591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1243c741-b4be-40bf-8ede-8cd558abfafd None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:24.102159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1243c741-b4be-40bf-8ede-8cd558abfafd None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:24.145968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53040bc3-25e8-45b6-b572-b84a05460c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:24.146594 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53040bc3-25e8-45b6-b572-b84a05460c74 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:24.146878 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53040bc3-25e8-45b6-b572-b84a05460c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:24.147152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53040bc3-25e8-45b6-b572-b84a05460c74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:24.150476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:24.163080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53040bc3-25e8-45b6-b572-b84a05460c74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:24.167807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-53040bc3-25e8-45b6-b572-b84a05460c74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:24.167807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:24.167807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:24.167807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:24.167807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:24.167807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:24.167807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:24.167807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:24.167807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:24.167807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:24.167807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:24.167807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:24.167807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:24.167807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:24.167807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:24.167807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:24.168657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:24.168657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:24.168657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:24.168657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:24.168657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:24.168657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:24.168657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:24.168657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:24.168657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:24.168657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:24.168657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:24.168657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:24.168657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:24.168657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:24.168657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:24.168657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:24.169451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:24.169451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:24.169451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:24.169451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:24.169451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:24.169451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:24.169451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:24.169451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:24.169451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:24.169451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:24.169451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:24.169451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:24.171506 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 245/483] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:24.176565 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532f46d-6501-425e-bdbf-9e0ad925568c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:24.309052 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532f46d-6501-425e-bdbf-9e0ad925568c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:24.328270 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:24.356453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfac9b2-1858-41fe-be18-52cdfd8e8ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:24.356989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfac9b2-1858-41fe-be18-52cdfd8e8ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:24.357262 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfac9b2-1858-41fe-be18-52cdfd8e8ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:24.357528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfac9b2-1858-41fe-be18-52cdfd8e8ef5 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:24.358190 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfac9b2-1858-41fe-be18-52cdfd8e8ef5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:24.358521 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfac9b2-1858-41fe-be18-52cdfd8e8ef5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:24.359900 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfac9b2-1858-41fe-be18-52cdfd8e8ef5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:24.373850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 239/484] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:13:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:24.379114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53040bc3-25e8-45b6-b572-b84a05460c74 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:24.505385 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa7ab1-3556-4738-af62-2de92d7f19d6 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:24.505693 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa7ab1-3556-4738-af62-2de92d7f19d6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:24.505950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa7ab1-3556-4738-af62-2de92d7f19d6 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:24.507537 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 240/485] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:24.512596 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e432d-28d8-4a4e-8e6b-5c36770347eb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:24.512878 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e432d-28d8-4a4e-8e6b-5c36770347eb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:24.513148 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e432d-28d8-4a4e-8e6b-5c36770347eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:24.532280 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e55e432d-28d8-4a4e-8e6b-5c36770347eb None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:24.532456 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53040bc3-25e8-45b6-b572-b84a05460c74 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:24.533483 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e55e432d-28d8-4a4e-8e6b-5c36770347eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:24.576879 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9e076-4275-4888-9003-e13c9a89e439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:24.577512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9e076-4275-4888-9003-e13c9a89e439 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:24.577800 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9e076-4275-4888-9003-e13c9a89e439 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:24.578398 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9e076-4275-4888-9003-e13c9a89e439 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:24.594163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9e076-4275-4888-9003-e13c9a89e439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:24.595656 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9e076-4275-4888-9003-e13c9a89e439 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:24.611965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-72b9e076-4275-4888-9003-e13c9a89e439 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:24.611965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:24.611965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:24.611965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:24.611965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:24.611965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:24.611965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:24.611965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:24.611965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:24.611965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:24.611965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:24.611965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:24.611965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:24.611965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:24.611965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:24.611965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:24.613001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:24.613001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:24.613001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:24.613001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:24.613001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:24.613001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:24.613001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:24.613001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:24.613001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:24.613001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:24.613001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:24.613001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:24.613001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:24.613001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:24.613001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:24.613001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:24.613001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:24.613808 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:24.613808 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:24.613808 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:24.613808 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 246/486] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:24.618273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 241/487] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:24.618387 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd7d0c-5028-40f6-af4d-d525c16d99a7 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:24.618387 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd7d0c-5028-40f6-af4d-d525c16d99a7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:24.618387 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd7d0c-5028-40f6-af4d-d525c16d99a7 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:24.618672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 247/488] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:24.623531 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e432d-28d8-4a4e-8e6b-5c36770347eb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:24.626972 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cb4d3-0704-41fa-b670-d24206f3c958 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:24.627252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cb4d3-0704-41fa-b670-d24206f3c958 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:24.627524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cb4d3-0704-41fa-b670-d24206f3c958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:24.655679 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-fc6cb4d3-0704-41fa-b670-d24206f3c958 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:24.657508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-fc6cb4d3-0704-41fa-b670-d24206f3c958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:24.710520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:24.744802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e432d-28d8-4a4e-8e6b-5c36770347eb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:24.748556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 248/489] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:24.749577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cb4d3-0704-41fa-b670-d24206f3c958 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:24.763391 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:24.804549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c066e5c-c167-4ce5-9156-f4c576312b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:24.805253 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c066e5c-c167-4ce5-9156-f4c576312b61 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:24.805606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c066e5c-c167-4ce5-9156-f4c576312b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:24.805971 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c066e5c-c167-4ce5-9156-f4c576312b61 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:24.806718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c066e5c-c167-4ce5-9156-f4c576312b61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:24.807090 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c066e5c-c167-4ce5-9156-f4c576312b61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:24.808648 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c066e5c-c167-4ce5-9156-f4c576312b61 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:24.820898 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 242/490] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:24.821913 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c066e5c-c167-4ce5-9156-f4c576312b61 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:24.879203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cb4d3-0704-41fa-b670-d24206f3c958 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:24.924782 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956237d0-5cc6-4094-978a-f88a54137536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:24.925503 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956237d0-5cc6-4094-978a-f88a54137536 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:24.925879 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956237d0-5cc6-4094-978a-f88a54137536 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:24.926222 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956237d0-5cc6-4094-978a-f88a54137536 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:24.937698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956237d0-5cc6-4094-978a-f88a54137536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:24.941789 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c066e5c-c167-4ce5-9156-f4c576312b61 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:24.942723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-956237d0-5cc6-4094-978a-f88a54137536 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:24.942723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:24.942723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:24.942723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:24.942723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:24.942723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:24.942723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:24.942723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:24.942723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:24.942723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:24.942723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:24.942723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:24.942723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:24.942723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:24.942723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:24.942723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:24.943548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:24.943548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:24.943548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:24.943548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:24.943548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:24.943548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:24.943548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:24.943548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:24.943548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:24.943548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:24.943548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:24.943548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:24.943548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:24.943548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:24.943548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:24.943548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:24.944320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:24.944320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:24.944320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:24.944320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:24.944320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:24.944320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:24.944320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:24.944320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:24.944320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:24.944320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:24.944320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:24.944320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:24.944320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 249/491] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:24.944940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956237d0-5cc6-4094-978a-f88a54137536 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:24.984173 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84939a-5214-4968-a317-24ca52c610be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:24.984901 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84939a-5214-4968-a317-24ca52c610be None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:24.985272 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84939a-5214-4968-a317-24ca52c610be None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:24.985617 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84939a-5214-4968-a317-24ca52c610be None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:24.989923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:24.998223 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da84939a-5214-4968-a317-24ca52c610be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:25.003811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-da84939a-5214-4968-a317-24ca52c610be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:25.003811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:25.003811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:25.003811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:25.003811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:25.003811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:25.003811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:25.003811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:25.003811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:25.003811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:25.003811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:25.003811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:25.003811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:25.003811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:25.003811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:25.003811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:25.004660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:25.004660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:25.004660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:25.004660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:25.004660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:25.004660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:25.004660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:25.004660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:25.004660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:25.004660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:25.004660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:25.004660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:25.004660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:25.004660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:25.004660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:25.004660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:25.005489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:25.005489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:25.005489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:25.005489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:25.005489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:25.005489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:25.005489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:25.005489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:25.005489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:25.005489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:25.005489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:25.005489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:25.005489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 243/492] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:25.010788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84939a-5214-4968-a317-24ca52c610be None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:25.042454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:25.076477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956237d0-5cc6-4094-978a-f88a54137536 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:25.123425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f496c-eafe-43af-b2f2-dd9344370fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:25.124962 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f496c-eafe-43af-b2f2-dd9344370fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:25.125418 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f496c-eafe-43af-b2f2-dd9344370fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:25.125875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f496c-eafe-43af-b2f2-dd9344370fc1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:25.126918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84939a-5214-4968-a317-24ca52c610be None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:25.135312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f496c-eafe-43af-b2f2-dd9344370fc1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:25.137452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f496c-eafe-43af-b2f2-dd9344370fc1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:25.154957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-035f496c-eafe-43af-b2f2-dd9344370fc1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:13:25.154957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:25.154957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:25.154957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:25.154957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:25.154957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:25.154957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:25.154957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:25.154957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:25.154957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:25.154957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:25.154957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:25.154957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:25.154957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:25.154957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:25.154957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:25.155840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:25.155840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:25.155840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:25.155840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:25.155840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:25.155840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:25.155840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:25.155840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:25.155840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:25.155840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:25.155840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:25.155840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:25.155840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:25.155840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:25.155840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:25.155840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:25.155840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:25.155840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:25.156684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:25.156684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:25.156684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:25.156684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:25.156684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:25.156684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:25.156684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:25.156684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:25.156684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:13:25.156684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:25.156684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 250/493] 10.222.0.39 () {56 vars in 1093 bytes} [Mon Aug 8 02:13:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:25.157392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f496c-eafe-43af-b2f2-dd9344370fc1 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:25.176932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffa492-d108-4bc2-a8a7-a905b9c4687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:25.178002 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffa492-d108-4bc2-a8a7-a905b9c4687f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:25.178519 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffa492-d108-4bc2-a8a7-a905b9c4687f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:25.179264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffa492-d108-4bc2-a8a7-a905b9c4687f None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:25.180065 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ffa492-d108-4bc2-a8a7-a905b9c4687f None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:25.180508 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ffa492-d108-4bc2-a8a7-a905b9c4687f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:25.182594 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ffa492-d108-4bc2-a8a7-a905b9c4687f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:25.195156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 244/494] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:13:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:25.196197 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffa492-d108-4bc2-a8a7-a905b9c4687f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:25.262964 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:25.270171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f496c-eafe-43af-b2f2-dd9344370fc1 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:25.310580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c006ac5e-5604-4e0b-92fb-a6c7cb94bead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:25.311195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006ac5e-5604-4e0b-92fb-a6c7cb94bead None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:25.311573 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006ac5e-5604-4e0b-92fb-a6c7cb94bead None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:25.311954 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006ac5e-5604-4e0b-92fb-a6c7cb94bead None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:25.312776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c006ac5e-5604-4e0b-92fb-a6c7cb94bead None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:25.313267 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c006ac5e-5604-4e0b-92fb-a6c7cb94bead None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:25.315007 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c006ac5e-5604-4e0b-92fb-a6c7cb94bead None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:25.319222 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:25.329761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 251/495] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:13:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:25.331658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c006ac5e-5604-4e0b-92fb-a6c7cb94bead None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:25.334469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffa492-d108-4bc2-a8a7-a905b9c4687f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:25.371869 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7f282-e749-4cec-bce6-1a10613ed7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:25.372463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7f282-e749-4cec-bce6-1a10613ed7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:25.372738 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7f282-e749-4cec-bce6-1a10613ed7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:25.372999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7f282-e749-4cec-bce6-1a10613ed7b3 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:25.373715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7f282-e749-4cec-bce6-1a10613ed7b3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:25.374013 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7f282-e749-4cec-bce6-1a10613ed7b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:25.375489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7f282-e749-4cec-bce6-1a10613ed7b3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:25.415876 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 245/496] 10.222.0.39 () {56 vars in 1133 bytes} [Mon Aug 8 02:13:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:25.415876 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7f282-e749-4cec-bce6-1a10613ed7b3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:25.449408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c006ac5e-5604-4e0b-92fb-a6c7cb94bead None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:25.482732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5fe74-79ce-4afe-b047-cb10c29f30d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:25.483350 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5fe74-79ce-4afe-b047-cb10c29f30d1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:25.483626 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5fe74-79ce-4afe-b047-cb10c29f30d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:25.483899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5fe74-79ce-4afe-b047-cb10c29f30d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:25.494362 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5fe74-79ce-4afe-b047-cb10c29f30d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:25.495798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5fe74-79ce-4afe-b047-cb10c29f30d1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:25.509918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-6cc5fe74-79ce-4afe-b047-cb10c29f30d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:25.509918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:25.509918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:25.509918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:25.509918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:25.509918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:25.509918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:25.509918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:25.509918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:25.509918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:25.509918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:25.509918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:25.509918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:25.509918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:25.509918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:25.509918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:25.511065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:25.511065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:25.511065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:25.511065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:25.511065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:25.511065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:25.511065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:25.511065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:25.511065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:25.511065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:25.511065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:25.511065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:25.511065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:25.511065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:25.511065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:25.511065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:25.511065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:25.511908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:25.511908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:25.511908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:25.511908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 252/497] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:25.512293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e776dcd-107a-462f-8c59-914a39400ecb None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:25.512578 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e776dcd-107a-462f-8c59-914a39400ecb None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:25.512852 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e776dcd-107a-462f-8c59-914a39400ecb None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:25.514193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 253/498] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:25.514855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e776dcd-107a-462f-8c59-914a39400ecb None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:25.524392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7f282-e749-4cec-bce6-1a10613ed7b3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:25.527529 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:25.558499 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a5183-6845-4469-b252-b5fa1768512f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:25.559112 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a5183-6845-4469-b252-b5fa1768512f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:25.559388 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a5183-6845-4469-b252-b5fa1768512f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:25.559915 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a5183-6845-4469-b252-b5fa1768512f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:25.570155 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a5183-6845-4469-b252-b5fa1768512f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:25.571638 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a5183-6845-4469-b252-b5fa1768512f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:25.578628 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:25.587704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f82a5183-6845-4469-b252-b5fa1768512f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:25.587704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:25.587704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:25.587704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:25.587704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:25.587704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:25.587704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:25.587704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:25.587704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:25.587704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:25.587704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:25.587704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:25.587704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:25.587704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:25.587704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:25.587704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:25.588762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:25.588762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:25.588762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:25.588762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:25.588762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:25.588762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:25.588762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:25.588762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:25.588762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:25.588762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:25.588762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:25.588762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:25.588762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:25.588762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:25.588762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:25.588762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:25.588762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:25.589611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:25.589611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:25.589611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:25.589611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 246/499] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:25.590149 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b79615-d175-42b6-ace0-c84aaa2cc472 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:25.590475 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b79615-d175-42b6-ace0-c84aaa2cc472 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:25.590749 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b79615-d175-42b6-ace0-c84aaa2cc472 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:25.591964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 247/500] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:25.592651 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b79615-d175-42b6-ace0-c84aaa2cc472 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:25.621546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e776dcd-107a-462f-8c59-914a39400ecb None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:25.655597 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54852857-234f-4cb5-9132-a6ee2ef229c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:25.656195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54852857-234f-4cb5-9132-a6ee2ef229c2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:25.656478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54852857-234f-4cb5-9132-a6ee2ef229c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:25.656752 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54852857-234f-4cb5-9132-a6ee2ef229c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:25.699262 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b79615-d175-42b6-ace0-c84aaa2cc472 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:25.723552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54852857-234f-4cb5-9132-a6ee2ef229c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:25.724998 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54852857-234f-4cb5-9132-a6ee2ef229c2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:25.731212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d951d67-0c44-40f7-95bc-5256c3c174b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:25.731806 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d951d67-0c44-40f7-95bc-5256c3c174b2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:25.732085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d951d67-0c44-40f7-95bc-5256c3c174b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:25.732364 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d951d67-0c44-40f7-95bc-5256c3c174b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:25.732996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d951d67-0c44-40f7-95bc-5256c3c174b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:25.733298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d951d67-0c44-40f7-95bc-5256c3c174b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:25.734699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d951d67-0c44-40f7-95bc-5256c3c174b2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:25.735661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:25.746765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 248/501] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:25.747929 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c010e-3638-4393-b5b2-283bd2e7d65a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:25.748209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c010e-3638-4393-b5b2-283bd2e7d65a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:25.748478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c010e-3638-4393-b5b2-283bd2e7d65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:25.765868 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-dc7c010e-3638-4393-b5b2-283bd2e7d65a None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:25.767093 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dc7c010e-3638-4393-b5b2-283bd2e7d65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:25.818616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:25.823667 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 254/502] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:25.824506 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54852857-234f-4cb5-9132-a6ee2ef229c2 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:25.833123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 249/503] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:25.834530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b25ef-b605-4e55-b57c-70492958b489 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:25.834813 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b25ef-b605-4e55-b57c-70492958b489 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:25.835109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b25ef-b605-4e55-b57c-70492958b489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:25.852159 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-179b25ef-b605-4e55-b57c-70492958b489 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:25.853338 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-179b25ef-b605-4e55-b57c-70492958b489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:25.918081 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 250/504] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:25.918926 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b25ef-b605-4e55-b57c-70492958b489 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:25.936955 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54852857-234f-4cb5-9132-a6ee2ef229c2 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:25.948445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:25.972840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17708b6a-335f-4104-8589-d71436e9f6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:25.973875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17708b6a-335f-4104-8589-d71436e9f6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:25.974389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17708b6a-335f-4104-8589-d71436e9f6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:25.974850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17708b6a-335f-4104-8589-d71436e9f6e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:25.975984 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17708b6a-335f-4104-8589-d71436e9f6e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:25.976499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17708b6a-335f-4104-8589-d71436e9f6e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:25.978983 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17708b6a-335f-4104-8589-d71436e9f6e0 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:25.992725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 255/505] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:25.993537 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17708b6a-335f-4104-8589-d71436e9f6e0 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:26.022565 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b25ef-b605-4e55-b57c-70492958b489 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:26.039969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:26.058155 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7fbe2-23c1-4392-a402-0f0edf07d36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:26.058814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7fbe2-23c1-4392-a402-0f0edf07d36e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:26.059088 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7fbe2-23c1-4392-a402-0f0edf07d36e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:26.059365 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7fbe2-23c1-4392-a402-0f0edf07d36e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:26.066740 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7fbe2-23c1-4392-a402-0f0edf07d36e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:26.068389 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7fbe2-23c1-4392-a402-0f0edf07d36e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:26.081975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-06f7fbe2-23c1-4392-a402-0f0edf07d36e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:13:26.081975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:26.081975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:26.081975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:26.081975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:26.081975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:26.081975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:26.081975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:26.081975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:26.081975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:26.081975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:26.081975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:26.081975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:26.081975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:26.081975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:26.081975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:26.082918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:26.082918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:26.082918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:26.082918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:26.082918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:26.082918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:26.082918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:26.082918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:26.082918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:26.082918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:26.082918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:26.082918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:26.082918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:26.082918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:26.082918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:26.082918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:26.082918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:26.082918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:26.083775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:26.083775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:26.083775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:26.083775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:26.083775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:26.083775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:26.083775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:26.083775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:26.083775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:13:26.083775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:26.083775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 251/506] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:26.084316 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7fbe2-23c1-4392-a402-0f0edf07d36e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:26.100124 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17708b6a-335f-4104-8589-d71436e9f6e0 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:26.133249 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b3818-88bc-4fb5-bfa4-5faf1c6864ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:26.133861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b3818-88bc-4fb5-bfa4-5faf1c6864ff None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:26.134139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b3818-88bc-4fb5-bfa4-5faf1c6864ff None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:26.134450 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b3818-88bc-4fb5-bfa4-5faf1c6864ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:26.144764 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b3818-88bc-4fb5-bfa4-5faf1c6864ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:26.146209 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b3818-88bc-4fb5-bfa4-5faf1c6864ff None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:26.159754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-5d8b3818-88bc-4fb5-bfa4-5faf1c6864ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:26.159754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:26.159754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:26.159754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:26.159754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:26.159754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:26.159754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:26.159754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:26.159754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:26.159754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:26.159754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:26.159754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:26.159754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:26.159754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:26.159754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:26.159754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:26.160634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:26.160634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:26.160634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:26.160634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:26.160634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:26.160634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:26.160634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:26.160634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:26.160634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:26.160634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:26.160634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:26.160634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:26.160634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:26.160634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:26.160634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:26.160634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:26.160634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:26.162002 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:26.162002 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:26.162002 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:26.162002 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 256/507] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:26.162225 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b3818-88bc-4fb5-bfa4-5faf1c6864ff None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:26.187527 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:26.187676 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7fbe2-23c1-4392-a402-0f0edf07d36e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:26.220505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667589af-57b4-44e4-843d-b7831cf41bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:26.221118 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667589af-57b4-44e4-843d-b7831cf41bea None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:26.221395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667589af-57b4-44e4-843d-b7831cf41bea None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:26.221663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667589af-57b4-44e4-843d-b7831cf41bea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:26.231924 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667589af-57b4-44e4-843d-b7831cf41bea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:26.236981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-667589af-57b4-44e4-843d-b7831cf41bea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:26.236981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:26.236981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:26.236981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:26.236981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:26.236981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:26.236981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:26.236981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:26.236981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:26.236981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:26.236981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:26.236981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:26.236981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:26.236981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:26.236981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:26.236981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:26.237844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:26.237844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:26.237844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:26.237844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:26.237844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:26.237844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:26.237844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:26.237844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:26.237844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:26.237844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:26.237844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:26.237844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:26.237844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:26.237844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:26.237844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:26.237844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:26.238889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:26.238889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:26.238889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:26.238889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:26.238889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:26.238889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:26.238889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:26.238889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:26.238889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:26.238889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:26.238889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:26.238889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:26.238889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 252/508] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:26.240656 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667589af-57b4-44e4-843d-b7831cf41bea None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:26.265266 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b3818-88bc-4fb5-bfa4-5faf1c6864ff None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:26.280217 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:26.307492 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c4309-f042-4061-86a1-7352d05f3087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:26.308125 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c4309-f042-4061-86a1-7352d05f3087 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:26.308410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c4309-f042-4061-86a1-7352d05f3087 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:26.308675 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c4309-f042-4061-86a1-7352d05f3087 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:26.319921 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c4309-f042-4061-86a1-7352d05f3087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:26.321376 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c4309-f042-4061-86a1-7352d05f3087 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:26.335332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-cb6c4309-f042-4061-86a1-7352d05f3087 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:26.335332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:26.335332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:26.335332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:26.335332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:26.335332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:26.335332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:26.335332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:26.335332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:26.335332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:26.335332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:26.335332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:26.335332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:26.335332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:26.335332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:26.335332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:26.336407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:26.336407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:26.336407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:26.336407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:26.336407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:26.336407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:26.336407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:26.336407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:26.336407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:26.336407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:26.336407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:26.336407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:26.336407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:26.336407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:26.336407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:26.336407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:26.336407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:26.337425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:26.337425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:26.337425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:26.337425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 257/509] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:26.337425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c4309-f042-4061-86a1-7352d05f3087 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:26.349758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667589af-57b4-44e4-843d-b7831cf41bea None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:26.384321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc7801-d269-4df6-b3e0-2a095b81a87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:26.384943 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc7801-d269-4df6-b3e0-2a095b81a87d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:26.385224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc7801-d269-4df6-b3e0-2a095b81a87d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:26.385499 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc7801-d269-4df6-b3e0-2a095b81a87d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:26.392821 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc7801-d269-4df6-b3e0-2a095b81a87d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:26.394466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc7801-d269-4df6-b3e0-2a095b81a87d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:26.405733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-97cc7801-d269-4df6-b3e0-2a095b81a87d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 02:13:26.405733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:26.405733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:26.405733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:26.405733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:26.405733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:26.405733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:26.405733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:26.405733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:26.405733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:26.405733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:26.405733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:26.405733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:26.405733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:26.405733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:26.405733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:26.406822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:26.406822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:26.406822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:26.406822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:26.406822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:26.406822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:26.406822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:26.406822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:26.406822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:26.406822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:26.406822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:26.406822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:26.406822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:26.406822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:26.406822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:26.406822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:26.406822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:26.406822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:26.407739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:26.407739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:26.407739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:26.407739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:26.407739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:26.407739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:26.407739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:26.407739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:26.407739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 02:13:26.407739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:26.407739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 253/510] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:26.407739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc7801-d269-4df6-b3e0-2a095b81a87d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:26.414449 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:26.441447 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c4309-f042-4061-86a1-7352d05f3087 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:26.476573 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d660d-804f-49c7-aa18-a9387fc7eac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:26.477161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d660d-804f-49c7-aa18-a9387fc7eac1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:26.477433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d660d-804f-49c7-aa18-a9387fc7eac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:26.477700 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d660d-804f-49c7-aa18-a9387fc7eac1 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:26.478440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d660d-804f-49c7-aa18-a9387fc7eac1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:26.478744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d660d-804f-49c7-aa18-a9387fc7eac1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:26.480150 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d660d-804f-49c7-aa18-a9387fc7eac1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:26.506120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:26.509907 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc7801-d269-4df6-b3e0-2a095b81a87d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:26.516060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 258/511] 10.222.0.39 () {56 vars in 1137 bytes} [Mon Aug 8 02:13:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:26.516697 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d660d-804f-49c7-aa18-a9387fc7eac1 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:26.544342 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63bdc65-ea35-400c-8c68-14f9840c1bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:26.544929 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bdc65-ea35-400c-8c68-14f9840c1bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:26.545205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bdc65-ea35-400c-8c68-14f9840c1bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:26.545469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bdc65-ea35-400c-8c68-14f9840c1bf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:26.546097 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63bdc65-ea35-400c-8c68-14f9840c1bf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:26.546430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63bdc65-ea35-400c-8c68-14f9840c1bf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:26.547784 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63bdc65-ea35-400c-8c68-14f9840c1bf2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:26.558992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 254/512] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:26.560012 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63bdc65-ea35-400c-8c68-14f9840c1bf2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:26.616885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d660d-804f-49c7-aa18-a9387fc7eac1 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:26.647239 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:26.651871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78365bbb-edf8-423e-8207-8b11a2d718d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:26.652467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78365bbb-edf8-423e-8207-8b11a2d718d5 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:26.652745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78365bbb-edf8-423e-8207-8b11a2d718d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:26.653014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78365bbb-edf8-423e-8207-8b11a2d718d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:26.653668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78365bbb-edf8-423e-8207-8b11a2d718d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:26.653972 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78365bbb-edf8-423e-8207-8b11a2d718d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:26.655385 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78365bbb-edf8-423e-8207-8b11a2d718d5 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:26.663832 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 259/513] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:13:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:26.664029 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63bdc65-ea35-400c-8c68-14f9840c1bf2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:26.664618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78365bbb-edf8-423e-8207-8b11a2d718d5 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:26.699107 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252a149-730b-47a1-a263-5ead38935aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:26.699710 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252a149-730b-47a1-a263-5ead38935aad None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:26.699986 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252a149-730b-47a1-a263-5ead38935aad None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:26.700263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252a149-730b-47a1-a263-5ead38935aad None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:26.700897 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252a149-730b-47a1-a263-5ead38935aad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:26.701198 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252a149-730b-47a1-a263-5ead38935aad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:26.702602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252a149-730b-47a1-a263-5ead38935aad None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:26.714336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 255/514] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:26.715218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252a149-730b-47a1-a263-5ead38935aad None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:26.734935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:26.768332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78365bbb-edf8-423e-8207-8b11a2d718d5 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:26.802531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaff17d-d4df-46bd-949d-ac40e04d28a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:26.803127 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaff17d-d4df-46bd-949d-ac40e04d28a5 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:26.803410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaff17d-d4df-46bd-949d-ac40e04d28a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:26.803684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaff17d-d4df-46bd-949d-ac40e04d28a5 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:26.804406 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaff17d-d4df-46bd-949d-ac40e04d28a5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:26.804722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaff17d-d4df-46bd-949d-ac40e04d28a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:26.806151 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaff17d-d4df-46bd-949d-ac40e04d28a5 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:26.814915 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252a149-730b-47a1-a263-5ead38935aad None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:26.823536 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 260/515] 10.222.0.39 () {56 vars in 1137 bytes} [Mon Aug 8 02:13:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:26.824405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaff17d-d4df-46bd-949d-ac40e04d28a5 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:26.853574 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd0070-a37c-4b6e-a745-61b00fb2b0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:26.854178 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd0070-a37c-4b6e-a745-61b00fb2b0cf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:26.854489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd0070-a37c-4b6e-a745-61b00fb2b0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:26.854758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd0070-a37c-4b6e-a745-61b00fb2b0cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:26.877276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:26.921201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd0070-a37c-4b6e-a745-61b00fb2b0cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:26.922696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd0070-a37c-4b6e-a745-61b00fb2b0cf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:26.924080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaff17d-d4df-46bd-949d-ac40e04d28a5 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:26.943156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:26.959182 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4684464-d0a1-4aa6-8ecb-ac06e4fafcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:26.959822 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4684464-d0a1-4aa6-8ecb-ac06e4fafcd7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:26.960095 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4684464-d0a1-4aa6-8ecb-ac06e4fafcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:26.960365 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4684464-d0a1-4aa6-8ecb-ac06e4fafcd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:26.966965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4684464-d0a1-4aa6-8ecb-ac06e4fafcd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:26.968593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4684464-d0a1-4aa6-8ecb-ac06e4fafcd7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:26.980856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-a4684464-d0a1-4aa6-8ecb-ac06e4fafcd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:26.980856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:26.980856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:26.980856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:26.980856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:26.980856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:26.980856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:26.980856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:26.980856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:26.980856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:26.980856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:26.980856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:26.980856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:26.980856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:26.980856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:26.981915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:26.981915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:26.981915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:26.981915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:26.981915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:26.981915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:26.981915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:26.981915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:26.981915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:26.981915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:26.981915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:26.981915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:26.981915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:26.981915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:26.981915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:26.981915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:26.981915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:26.982923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:26.982923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:26.982923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:26.982923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:26.982923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:26.982923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:26.982923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:26.982923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:26.982923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 261/516] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:26.983490 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4684464-d0a1-4aa6-8ecb-ac06e4fafcd7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:27.027654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 256/517] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:27.028404 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd0070-a37c-4b6e-a745-61b00fb2b0cf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:27.087784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4684464-d0a1-4aa6-8ecb-ac06e4fafcd7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:27.099980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:27.122014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107fcce-1f4c-4ff1-a90b-b1f8cab91968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:27.122661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107fcce-1f4c-4ff1-a90b-b1f8cab91968 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:27.122939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107fcce-1f4c-4ff1-a90b-b1f8cab91968 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:27.123210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107fcce-1f4c-4ff1-a90b-b1f8cab91968 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:27.125318 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd0070-a37c-4b6e-a745-61b00fb2b0cf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:27.133974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107fcce-1f4c-4ff1-a90b-b1f8cab91968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:27.135457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107fcce-1f4c-4ff1-a90b-b1f8cab91968 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:27.142433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:27.150120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-6107fcce-1f4c-4ff1-a90b-b1f8cab91968 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:27.150120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:27.150120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:27.150120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:27.150120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:27.150120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:27.150120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:27.150120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:27.150120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:27.150120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:27.150120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:27.150120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:27.150120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:27.150120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:27.150120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:27.150120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:27.151223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:27.151223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:27.151223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:27.151223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:27.151223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:27.151223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:27.151223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:27.151223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:27.151223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:27.151223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:27.151223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:27.151223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:27.151223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:27.151223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:27.151223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:27.151223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:27.151223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:27.152247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:27.152247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:27.152247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:27.152247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 262/518] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:27.152247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107fcce-1f4c-4ff1-a90b-b1f8cab91968 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:27.164534 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68baa04f-2e62-41ba-a269-90b98f69bd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:27.165145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68baa04f-2e62-41ba-a269-90b98f69bd75 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:27.165435 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68baa04f-2e62-41ba-a269-90b98f69bd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:27.165705 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68baa04f-2e62-41ba-a269-90b98f69bd75 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:27.166371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68baa04f-2e62-41ba-a269-90b98f69bd75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:27.167812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68baa04f-2e62-41ba-a269-90b98f69bd75 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:27.189755 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 257/519] 10.222.0.39 () {60 vars in 1132 bytes} [Mon Aug 8 02:13:27 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:27.190459 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68baa04f-2e62-41ba-a269-90b98f69bd75 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:27.272664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107fcce-1f4c-4ff1-a90b-b1f8cab91968 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:27.309067 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68baa04f-2e62-41ba-a269-90b98f69bd75 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:27.311785 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774107d8-6dbb-4307-8815-a2f51fb0be91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:27.312410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774107d8-6dbb-4307-8815-a2f51fb0be91 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:27.312691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774107d8-6dbb-4307-8815-a2f51fb0be91 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:27.312971 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774107d8-6dbb-4307-8815-a2f51fb0be91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:27.349380 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:27.354477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1aa0c-9060-496b-8925-a327699ba62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:27.355089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1aa0c-9060-496b-8925-a327699ba62d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:27.355370 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1aa0c-9060-496b-8925-a327699ba62d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:27.355644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1aa0c-9060-496b-8925-a327699ba62d None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:27.356346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e1aa0c-9060-496b-8925-a327699ba62d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:27.356651 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e1aa0c-9060-496b-8925-a327699ba62d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:27.358177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e1aa0c-9060-496b-8925-a327699ba62d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:27.382333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 258/520] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:27.383675 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1aa0c-9060-496b-8925-a327699ba62d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:27.393855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774107d8-6dbb-4307-8815-a2f51fb0be91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:27.395409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774107d8-6dbb-4307-8815-a2f51fb0be91 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:27.428635 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:27.501238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1aa0c-9060-496b-8925-a327699ba62d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:27.507680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 263/521] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:27.512768 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774107d8-6dbb-4307-8815-a2f51fb0be91 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:27.540736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6ce1e-12e3-423e-9cc0-20bb226ffef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:27.541531 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6ce1e-12e3-423e-9cc0-20bb226ffef9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:27.541811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6ce1e-12e3-423e-9cc0-20bb226ffef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:27.542076 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6ce1e-12e3-423e-9cc0-20bb226ffef9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:27.553943 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6ce1e-12e3-423e-9cc0-20bb226ffef9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:27.555591 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6ce1e-12e3-423e-9cc0-20bb226ffef9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:27.571570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d1c6ce1e-12e3-423e-9cc0-20bb226ffef9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:27.571570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:27.571570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:27.571570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:27.571570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:27.571570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:27.571570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:27.571570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:27.571570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:27.571570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:27.571570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:27.571570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:27.571570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:27.571570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:27.571570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:27.571570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:27.572463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:27.572463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:27.572463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:27.572463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:27.572463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:27.572463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:27.572463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:27.572463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:27.572463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:27.572463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:27.572463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:27.572463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:27.572463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:27.572463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:27.572463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:27.572463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:27.572463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:27.573278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:27.573278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:27.573278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:27.577418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 259/522] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:27.578030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6ce1e-12e3-423e-9cc0-20bb226ffef9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:27.583479 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:27.625120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774107d8-6dbb-4307-8815-a2f51fb0be91 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:27.664323 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ec50e-c5ac-402f-9580-8b4215342a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:27.665137 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ec50e-c5ac-402f-9580-8b4215342a5d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:27.665526 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ec50e-c5ac-402f-9580-8b4215342a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:27.665865 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ec50e-c5ac-402f-9580-8b4215342a5d None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:27.666530 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ec50e-c5ac-402f-9580-8b4215342a5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:27.666828 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ec50e-c5ac-402f-9580-8b4215342a5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:27.668486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:27.668997 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ec50e-c5ac-402f-9580-8b4215342a5d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:27.686374 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 264/523] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:27.686374 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ec50e-c5ac-402f-9580-8b4215342a5d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:27.686621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6ce1e-12e3-423e-9cc0-20bb226ffef9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:27.720032 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83c1aa-a675-49f4-90c2-3f5ceeb483f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:27.720892 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83c1aa-a675-49f4-90c2-3f5ceeb483f7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:27.721949 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83c1aa-a675-49f4-90c2-3f5ceeb483f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:27.721949 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83c1aa-a675-49f4-90c2-3f5ceeb483f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:27.722564 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83c1aa-a675-49f4-90c2-3f5ceeb483f7 None admin] RBAC: Adding query filter params (role.id=838b081e674f4c0ab4a33bcf8e614dc5, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=c4238566a9c248ffbfa314f47f02670c) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:27.722974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83c1aa-a675-49f4-90c2-3f5ceeb483f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:27.725065 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83c1aa-a675-49f4-90c2-3f5ceeb483f7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:27.738698 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 260/524] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:27 2022] GET /identity/v3/role_assignments?user.id=c4238566a9c248ffbfa314f47f02670c&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=838b081e674f4c0ab4a33bcf8e614dc5 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:27.738698 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83c1aa-a675-49f4-90c2-3f5ceeb483f7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:27.810671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ec50e-c5ac-402f-9580-8b4215342a5d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:27.842825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:27.860075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edb4d2-3530-46ff-839e-96e286b52a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:27.860712 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edb4d2-3530-46ff-839e-96e286b52a0d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:27.861254 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edb4d2-3530-46ff-839e-96e286b52a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:27.861528 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edb4d2-3530-46ff-839e-96e286b52a0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:27.873220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83c1aa-a675-49f4-90c2-3f5ceeb483f7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:27.875987 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58edb4d2-3530-46ff-839e-96e286b52a0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:27.877578 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58edb4d2-3530-46ff-839e-96e286b52a0d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:27.894232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-58edb4d2-3530-46ff-839e-96e286b52a0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:27.894232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:27.894232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:27.894232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:27.894232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:27.894232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:27.894232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:27.894232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:27.894232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:27.894232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:27.894232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:27.894232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:27.894232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:27.894232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:27.894232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:27.894232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:27.896116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:27.896116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:27.896116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:27.896116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:27.896116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:27.896116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:27.896116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:27.896116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:27.896116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:27.896116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:27.896116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:27.896116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:27.896116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:27.896116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:27.896116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:27.896116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:27.896116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:27.897212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:27.897212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:27.897212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:27.897659 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 265/525] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:27.898753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edb4d2-3530-46ff-839e-96e286b52a0d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:27.926144 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaae962-fdf1-4473-86f3-c397d5df71e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:27.926978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaae962-fdf1-4473-86f3-c397d5df71e3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:27.927376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaae962-fdf1-4473-86f3-c397d5df71e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:27.927649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaae962-fdf1-4473-86f3-c397d5df71e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:27.928695 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaae962-fdf1-4473-86f3-c397d5df71e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:27.929047 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaae962-fdf1-4473-86f3-c397d5df71e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:27.931137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaae962-fdf1-4473-86f3-c397d5df71e3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:27.946896 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 261/526] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:27.948248 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaae962-fdf1-4473-86f3-c397d5df71e3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:27.963701 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:28.022776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edb4d2-3530-46ff-839e-96e286b52a0d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:28.058372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6a792-249b-4e35-b12e-33b7a9f2ab0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:28.058987 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6a792-249b-4e35-b12e-33b7a9f2ab0d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:28.059320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6a792-249b-4e35-b12e-33b7a9f2ab0d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:28.059839 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6a792-249b-4e35-b12e-33b7a9f2ab0d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:28.069335 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6a792-249b-4e35-b12e-33b7a9f2ab0d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:28.070924 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6a792-249b-4e35-b12e-33b7a9f2ab0d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:28.076398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaae962-fdf1-4473-86f3-c397d5df71e3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:28.083975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-03c6a792-249b-4e35-b12e-33b7a9f2ab0d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 02:13:28.083975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:28.083975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:28.083975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:28.083975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:28.083975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:28.083975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:28.083975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:28.083975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:28.083975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:28.083975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:28.083975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:28.083975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:28.083975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:28.083975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:28.083975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:28.084943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:28.084943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:28.084943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:28.084943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:28.084943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:28.084943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:28.084943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:28.084943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:28.084943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:28.084943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:28.084943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:28.084943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:28.084943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:28.084943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:28.084943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:28.084943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:28.084943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:28.084943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:28.086232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:28.086232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:28.086232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:28.086232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:28.086232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:28.086232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:28.086232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:28.086232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:28.086232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 02:13:28.086232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:28.086232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 266/527] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:28.086232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6a792-249b-4e35-b12e-33b7a9f2ab0d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:28.115076 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50faf68-8ddd-456e-bf25-1e48a2bf6740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:28.116104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50faf68-8ddd-456e-bf25-1e48a2bf6740 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:28.116594 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50faf68-8ddd-456e-bf25-1e48a2bf6740 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:28.117567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50faf68-8ddd-456e-bf25-1e48a2bf6740 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:28.118226 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50faf68-8ddd-456e-bf25-1e48a2bf6740 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:28.118608 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50faf68-8ddd-456e-bf25-1e48a2bf6740 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:28.120504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50faf68-8ddd-456e-bf25-1e48a2bf6740 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:28.133333 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:28.141602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 262/528] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:28.142534 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50faf68-8ddd-456e-bf25-1e48a2bf6740 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:28.220012 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6a792-249b-4e35-b12e-33b7a9f2ab0d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:28.236260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:28.259933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a712c-60ce-45ab-b307-507aadd065bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:28.260821 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a712c-60ce-45ab-b307-507aadd065bf None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:28.261096 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a712c-60ce-45ab-b307-507aadd065bf None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:28.261366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a712c-60ce-45ab-b307-507aadd065bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:28.270197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a712c-60ce-45ab-b307-507aadd065bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:28.272597 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50faf68-8ddd-456e-bf25-1e48a2bf6740 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:28.273022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a712c-60ce-45ab-b307-507aadd065bf None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:28.285435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-6f8a712c-60ce-45ab-b307-507aadd065bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:28.285435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:28.285435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:28.285435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:28.285435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:28.285435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:28.285435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:28.285435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:28.285435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:28.285435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:28.285435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:28.285435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:28.285435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:28.285435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:28.285435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:28.286922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:28.286922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:28.286922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:28.286922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:28.286922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:28.286922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:28.286922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:28.286922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:28.286922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:28.286922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:28.286922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:28.286922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:28.286922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:28.286922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:28.286922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:28.286922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:28.286922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:28.288457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:28.288457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:28.288457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:28.288457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:28.288457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:28.288457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:28.288457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:28.288457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:28.288457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 267/529] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:28.289708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a712c-60ce-45ab-b307-507aadd065bf None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:28.310782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e523c74-d605-486b-a6c8-27d52eccd8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:28.311382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e523c74-d605-486b-a6c8-27d52eccd8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:28.311655 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e523c74-d605-486b-a6c8-27d52eccd8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:28.311917 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e523c74-d605-486b-a6c8-27d52eccd8a0 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:28.312804 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e523c74-d605-486b-a6c8-27d52eccd8a0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:28.312981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e523c74-d605-486b-a6c8-27d52eccd8a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:28.314502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e523c74-d605-486b-a6c8-27d52eccd8a0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:28.333844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 263/530] 10.222.0.39 () {56 vars in 1139 bytes} [Mon Aug 8 02:13:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:28.334711 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e523c74-d605-486b-a6c8-27d52eccd8a0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:28.407645 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:28.416106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a712c-60ce-45ab-b307-507aadd065bf None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:28.438867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e523c74-d605-486b-a6c8-27d52eccd8a0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:28.457369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef89b2-32a1-43ee-9370-9f01a4654640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:28.458868 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef89b2-32a1-43ee-9370-9f01a4654640 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:28.459400 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef89b2-32a1-43ee-9370-9f01a4654640 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:28.459866 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef89b2-32a1-43ee-9370-9f01a4654640 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:28.467734 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef89b2-32a1-43ee-9370-9f01a4654640 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:28.469539 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef89b2-32a1-43ee-9370-9f01a4654640 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:28.477944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe43d1-85c3-4022-bed9-4085591db130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:28.478591 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe43d1-85c3-4022-bed9-4085591db130 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:28.478868 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe43d1-85c3-4022-bed9-4085591db130 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:28.479131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe43d1-85c3-4022-bed9-4085591db130 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:28.479811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe43d1-85c3-4022-bed9-4085591db130 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:28.480113 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe43d1-85c3-4022-bed9-4085591db130 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:28.481537 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe43d1-85c3-4022-bed9-4085591db130 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:28.484318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-64ef89b2-32a1-43ee-9370-9f01a4654640 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:28.484318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:28.484318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:28.484318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:28.484318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:28.484318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:28.484318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:28.484318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:28.484318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:28.484318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:28.484318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:28.484318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:28.484318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:28.484318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:28.484318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:28.485141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:28.485141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:28.485141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:28.485141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:28.485141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:28.485141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:28.485141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:28.485141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:28.485141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:28.485141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:28.485141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:28.485141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:28.485141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:28.485141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:28.485141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:28.485141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:28.485141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:28.486015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:28.486015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:28.486015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:28.486015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:28.486015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:28.486015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:28.486015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:28.486015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:28.488402 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 268/531] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:28.488809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:28.490677 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef89b2-32a1-43ee-9370-9f01a4654640 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:28.499394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 264/532] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:28.500512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe43d1-85c3-4022-bed9-4085591db130 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:28.606012 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe43d1-85c3-4022-bed9-4085591db130 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:28.606592 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef89b2-32a1-43ee-9370-9f01a4654640 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:28.640466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af162ae8-b7f0-4b93-92fe-e34a57efcc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:28.641281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af162ae8-b7f0-4b93-92fe-e34a57efcc43 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:28.641637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af162ae8-b7f0-4b93-92fe-e34a57efcc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:28.642002 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af162ae8-b7f0-4b93-92fe-e34a57efcc43 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:28.642865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af162ae8-b7f0-4b93-92fe-e34a57efcc43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:28.643216 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af162ae8-b7f0-4b93-92fe-e34a57efcc43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:28.644798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bd647-0a94-41aa-bf27-d199812c1d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:28.644947 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af162ae8-b7f0-4b93-92fe-e34a57efcc43 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:28.645800 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bd647-0a94-41aa-bf27-d199812c1d8d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:28.646266 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bd647-0a94-41aa-bf27-d199812c1d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:28.646699 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bd647-0a94-41aa-bf27-d199812c1d8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:28.662763 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 265/533] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:28.663719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af162ae8-b7f0-4b93-92fe-e34a57efcc43 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:28.673808 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:28.718776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:28.724349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bd647-0a94-41aa-bf27-d199812c1d8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:28.726354 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bd647-0a94-41aa-bf27-d199812c1d8d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:28.782436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af162ae8-b7f0-4b93-92fe-e34a57efcc43 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:28.818178 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0141f1-8255-4a3b-8ac8-b998dab9e009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:28.819041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0141f1-8255-4a3b-8ac8-b998dab9e009 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:28.819461 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0141f1-8255-4a3b-8ac8-b998dab9e009 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:28.819897 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0141f1-8255-4a3b-8ac8-b998dab9e009 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:28.820763 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0141f1-8255-4a3b-8ac8-b998dab9e009 None admin] RBAC: Adding query filter params (role.id=838b081e674f4c0ab4a33bcf8e614dc5, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=2812e59d0d9948879a465fe3b5b421e1) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:28.821214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0141f1-8255-4a3b-8ac8-b998dab9e009 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:28.822903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0141f1-8255-4a3b-8ac8-b998dab9e009 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:28.828828 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 266/534] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:28 2022] GET /identity/v3/role_assignments?user.id=2812e59d0d9948879a465fe3b5b421e1&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=838b081e674f4c0ab4a33bcf8e614dc5 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:28.829940 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0141f1-8255-4a3b-8ac8-b998dab9e009 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:28.846550 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 269/535] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:28.852634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bd647-0a94-41aa-bf27-d199812c1d8d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:28.948187 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0141f1-8255-4a3b-8ac8-b998dab9e009 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:28.974809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:28.990174 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:28.994985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4ab81-c2e6-4be1-8265-d9510b39bd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:28.996141 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4ab81-c2e6-4be1-8265-d9510b39bd2b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:28.997127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4ab81-c2e6-4be1-8265-d9510b39bd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:28.997700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4ab81-c2e6-4be1-8265-d9510b39bd2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:28.998743 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4ab81-c2e6-4be1-8265-d9510b39bd2b None admin] RBAC: Adding query filter params (role.id=838b081e674f4c0ab4a33bcf8e614dc5, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=7776a6a1faf34b369e399d83bb3e18c7) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:28.999284 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4ab81-c2e6-4be1-8265-d9510b39bd2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:29.001302 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4ab81-c2e6-4be1-8265-d9510b39bd2b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:29.010012 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 267/536] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:28 2022] GET /identity/v3/role_assignments?user.id=7776a6a1faf34b369e399d83bb3e18c7&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=838b081e674f4c0ab4a33bcf8e614dc5 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:29.057201 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bd647-0a94-41aa-bf27-d199812c1d8d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:29.097576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f56016-ad73-4870-8d8d-a589aafb24f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:29.099248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f56016-ad73-4870-8d8d-a589aafb24f5 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:29.099730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f56016-ad73-4870-8d8d-a589aafb24f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:29.100197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f56016-ad73-4870-8d8d-a589aafb24f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:29.112330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f56016-ad73-4870-8d8d-a589aafb24f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:29.114810 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f56016-ad73-4870-8d8d-a589aafb24f5 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:29.131546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-f7f56016-ad73-4870-8d8d-a589aafb24f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:29.131546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:29.131546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:29.131546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:29.131546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:29.131546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:29.131546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:29.131546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:29.131546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:29.131546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:29.131546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:29.131546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:29.131546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:29.131546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:29.131546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:29.131546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:29.132467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:29.132467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:29.132467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:29.132467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:29.132467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:29.132467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:29.132467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:29.132467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:29.132467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:29.132467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:29.132467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:29.132467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:29.132467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:29.132467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:29.132467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:29.132467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:29.132467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:29.133313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:29.133313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:29.133313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:29.135446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 270/537] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:29.143290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4ab81-c2e6-4be1-8265-d9510b39bd2b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:29.260528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4ab81-c2e6-4be1-8265-d9510b39bd2b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:29.295846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656a9ef-4b22-41e0-b2c5-5655ce5e4c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:29.296804 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656a9ef-4b22-41e0-b2c5-5655ce5e4c19 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:29.297283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656a9ef-4b22-41e0-b2c5-5655ce5e4c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:29.297774 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656a9ef-4b22-41e0-b2c5-5655ce5e4c19 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:29.298685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656a9ef-4b22-41e0-b2c5-5655ce5e4c19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:29.299272 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656a9ef-4b22-41e0-b2c5-5655ce5e4c19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:29.300925 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656a9ef-4b22-41e0-b2c5-5655ce5e4c19 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:29.314666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 268/538] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:29.323594 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f56016-ad73-4870-8d8d-a589aafb24f5 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:29.360910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba62fb2-1b81-4440-a0e7-57c3d7aeaea8 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:29.361126 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba62fb2-1b81-4440-a0e7-57c3d7aeaea8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:29.361341 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba62fb2-1b81-4440-a0e7-57c3d7aeaea8 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:29.366526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 269/539] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:29.371554 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f361d-e2f9-4a47-a4be-06d9fb37c295 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:29.371835 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f361d-e2f9-4a47-a4be-06d9fb37c295 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:29.372113 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f361d-e2f9-4a47-a4be-06d9fb37c295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:29.431307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-040f361d-e2f9-4a47-a4be-06d9fb37c295 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:29.432535 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-040f361d-e2f9-4a47-a4be-06d9fb37c295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:29.457406 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f56016-ad73-4870-8d8d-a589aafb24f5 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:29.486621 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:29.498632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:29.518033 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc943c-b0b8-4918-97f1-1927234702d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:29.518643 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc943c-b0b8-4918-97f1-1927234702d5 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:29.519054 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc943c-b0b8-4918-97f1-1927234702d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:29.519344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc943c-b0b8-4918-97f1-1927234702d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:29.527267 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc943c-b0b8-4918-97f1-1927234702d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:29.528924 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc943c-b0b8-4918-97f1-1927234702d5 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:29.542141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-aadc943c-b0b8-4918-97f1-1927234702d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:29.542141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:29.542141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:29.542141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:29.542141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:29.542141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:29.542141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:29.542141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:29.542141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:29.542141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:29.542141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:29.542141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:29.542141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:29.542141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:29.542141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:29.543022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:29.543022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:29.543022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:29.543022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:29.543022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:29.543022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:29.543022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:29.543022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:29.543022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:29.543022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:29.543022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:29.543022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:29.543022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:29.543022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:29.543022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:29.543022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:29.543022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:29.543837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:29.543837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:29.543837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:29.543837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:29.543837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:29.543837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:29.543837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:29.543837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:29.543837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 271/540] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:29.555546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc943c-b0b8-4918-97f1-1927234702d5 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:29.629678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 270/541] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:29.675644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f361d-e2f9-4a47-a4be-06d9fb37c295 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:29.688381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc943c-b0b8-4918-97f1-1927234702d5 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:29.758116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ee1d6-f0e7-4b9a-b923-e9dd4706e27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:29.758759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ee1d6-f0e7-4b9a-b923-e9dd4706e27b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:29.759039 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ee1d6-f0e7-4b9a-b923-e9dd4706e27b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:29.759313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ee1d6-f0e7-4b9a-b923-e9dd4706e27b None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:29.760016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ee1d6-f0e7-4b9a-b923-e9dd4706e27b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:29.766283 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ee1d6-f0e7-4b9a-b923-e9dd4706e27b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:29.766283 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ee1d6-f0e7-4b9a-b923-e9dd4706e27b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:29.784515 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 272/542] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:29.791059 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ee1d6-f0e7-4b9a-b923-e9dd4706e27b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:29.830304 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:29.871558 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f361d-e2f9-4a47-a4be-06d9fb37c295 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:29.941420 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ee1d6-f0e7-4b9a-b923-e9dd4706e27b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:29.946553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa4177c-8a69-4227-a5c1-ab8ab3f93197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:29.947262 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4177c-8a69-4227-a5c1-ab8ab3f93197 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:29.947616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4177c-8a69-4227-a5c1-ab8ab3f93197 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:29.947957 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4177c-8a69-4227-a5c1-ab8ab3f93197 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:29.962730 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4177c-8a69-4227-a5c1-ab8ab3f93197 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:29.969603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7aa4177c-8a69-4227-a5c1-ab8ab3f93197 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:29.969603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:29.969603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:29.969603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:29.969603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:29.969603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:29.969603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:29.969603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:29.969603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:29.969603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:29.969603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:29.969603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:29.969603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:29.969603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:29.969603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:29.969603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:29.970491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:29.970491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:29.970491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:29.970491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:29.970491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:29.970491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:29.970491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:29.970491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:29.970491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:29.970491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:29.970491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:29.970491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:29.970491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:29.970491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:29.970491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:29.970491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:29.971267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:29.971267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:29.971267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:29.971267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:29.971267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:29.971267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:29.971267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:29.971267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:29.971267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:29.971267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:29.971267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:29.971267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:29.974335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 271/543] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:29.975554 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9aba06-b641-4b59-b06c-85774062ad38 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:29.975835 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9aba06-b641-4b59-b06c-85774062ad38 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:29.976107 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9aba06-b641-4b59-b06c-85774062ad38 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:29.977351 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 272/544] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:29.978834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9aba06-b641-4b59-b06c-85774062ad38 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:29.986948 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ebc7d-0394-419b-ab9a-76d8f06d7f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:29.987553 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ebc7d-0394-419b-ab9a-76d8f06d7f5a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:29.987829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ebc7d-0394-419b-ab9a-76d8f06d7f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:29.988098 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ebc7d-0394-419b-ab9a-76d8f06d7f5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:29.988824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ebc7d-0394-419b-ab9a-76d8f06d7f5a None admin] RBAC: Adding query filter params (role.id=838b081e674f4c0ab4a33bcf8e614dc5, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=78506fea1c024b169a4b7ee86116658c) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:29.989170 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ebc7d-0394-419b-ab9a-76d8f06d7f5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:29.990640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ebc7d-0394-419b-ab9a-76d8f06d7f5a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:30.000759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 273/545] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:29 2022] GET /identity/v3/role_assignments?user.id=78506fea1c024b169a4b7ee86116658c&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=838b081e674f4c0ab4a33bcf8e614dc5 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:30.000759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1680f-862e-4fdb-9cf4-c00d06ec4a85 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:30.000759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1680f-862e-4fdb-9cf4-c00d06ec4a85 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:30.000759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1680f-862e-4fdb-9cf4-c00d06ec4a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:30.026526 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-74e1680f-862e-4fdb-9cf4-c00d06ec4a85 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:30.028027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-74e1680f-862e-4fdb-9cf4-c00d06ec4a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:30.042694 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:30.167246 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 274/546] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:30.177900 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9aba06-b641-4b59-b06c-85774062ad38 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:30.204941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1680f-862e-4fdb-9cf4-c00d06ec4a85 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:30.233618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08a648-3fe3-4480-ab46-0041a2cc5ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:30.234270 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08a648-3fe3-4480-ab46-0041a2cc5ace None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:30.234773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:30.235467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08a648-3fe3-4480-ab46-0041a2cc5ace None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:30.235747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08a648-3fe3-4480-ab46-0041a2cc5ace None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:30.236800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08a648-3fe3-4480-ab46-0041a2cc5ace None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:30.237420 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08a648-3fe3-4480-ab46-0041a2cc5ace None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:30.239108 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08a648-3fe3-4480-ab46-0041a2cc5ace None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:30.250156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 273/547] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:13:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:30.255768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08a648-3fe3-4480-ab46-0041a2cc5ace None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:30.378315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1680f-862e-4fdb-9cf4-c00d06ec4a85 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:30.414734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:30.426650 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7772d-1b93-476d-9774-593263ab6709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:30.426941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08a648-3fe3-4480-ab46-0041a2cc5ace None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:30.427777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7772d-1b93-476d-9774-593263ab6709 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:30.428257 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7772d-1b93-476d-9774-593263ab6709 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:30.428767 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7772d-1b93-476d-9774-593263ab6709 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:30.462993 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d52adc-6646-49ef-b765-f5fe4f52fbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:30.463617 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d52adc-6646-49ef-b765-f5fe4f52fbde None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:30.463896 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d52adc-6646-49ef-b765-f5fe4f52fbde None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:30.464168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d52adc-6646-49ef-b765-f5fe4f52fbde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:30.467188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7772d-1b93-476d-9774-593263ab6709 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:30.475310 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d52adc-6646-49ef-b765-f5fe4f52fbde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:30.476103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-40b7772d-1b93-476d-9774-593263ab6709 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:30.476103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:30.476103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:30.476103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:30.476103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:30.476103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:30.476103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:30.476103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:30.476103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:30.476103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:30.476103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:30.476103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:30.476103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:30.476103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:30.476103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:30.476103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:30.476974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:30.476974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:30.476974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:30.476974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:30.476974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:30.476974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:30.476974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:30.476974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:30.476974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:30.476974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:30.476974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:30.476974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:30.476974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:30.476974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:30.476974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:30.476974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:30.477737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:30.477737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:30.477737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:30.477737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:30.477737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:30.477737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:30.477737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:30.477737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:30.477737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:30.477737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:30.477737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:30.477737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:30.478381 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d52adc-6646-49ef-b765-f5fe4f52fbde None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:30.478450 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 275/548] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:30.486012 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7772d-1b93-476d-9774-593263ab6709 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:30.491201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-24d52adc-6646-49ef-b765-f5fe4f52fbde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:30.491201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:30.491201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:30.491201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:30.491201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:30.491201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:30.491201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:30.491201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:30.491201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:30.491201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:30.491201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:30.491201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:30.491201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:30.491201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:30.491201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:30.491201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:30.492063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:30.492063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:30.492063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:30.492063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:30.492063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:30.492063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:30.492063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:30.492063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:30.492063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:30.492063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:30.492063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:30.492063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:30.492063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:30.492063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:30.492063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:30.492063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:30.492063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:30.493512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:30.493512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:30.493512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:30.493512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 274/549] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:30.507221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d52adc-6646-49ef-b765-f5fe4f52fbde None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:30.626963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:30.635061 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d52adc-6646-49ef-b765-f5fe4f52fbde None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:30.668399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7772d-1b93-476d-9774-593263ab6709 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:30.671140 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7afeb9-190e-4630-af79-d85bbc03745c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:30.671273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7afeb9-190e-4630-af79-d85bbc03745c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:30.671555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7afeb9-190e-4630-af79-d85bbc03745c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:30.671898 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7afeb9-190e-4630-af79-d85bbc03745c None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:30.673318 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7afeb9-190e-4630-af79-d85bbc03745c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:30.673318 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7afeb9-190e-4630-af79-d85bbc03745c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:30.674399 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7afeb9-190e-4630-af79-d85bbc03745c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:30.683780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:30.693011 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 275/550] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:30.694610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7afeb9-190e-4630-af79-d85bbc03745c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:30.746270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3c223-b292-4a0e-ae87-e69bbe7e711f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:30.746936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3c223-b292-4a0e-ae87-e69bbe7e711f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:30.747215 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3c223-b292-4a0e-ae87-e69bbe7e711f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:30.747489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3c223-b292-4a0e-ae87-e69bbe7e711f None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:30.748156 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e3c223-b292-4a0e-ae87-e69bbe7e711f None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:30.748459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e3c223-b292-4a0e-ae87-e69bbe7e711f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:30.749917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e3c223-b292-4a0e-ae87-e69bbe7e711f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:30.763488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 276/551] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:13:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:30.771212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3c223-b292-4a0e-ae87-e69bbe7e711f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:30.809174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7afeb9-190e-4630-af79-d85bbc03745c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:30.845336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4895f4-1256-4e54-bd1d-7e2861e0f68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:30.846350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4895f4-1256-4e54-bd1d-7e2861e0f68f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:30.846836 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4895f4-1256-4e54-bd1d-7e2861e0f68f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:30.847665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4895f4-1256-4e54-bd1d-7e2861e0f68f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:30.856260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4895f4-1256-4e54-bd1d-7e2861e0f68f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:30.858041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4895f4-1256-4e54-bd1d-7e2861e0f68f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:30.872981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ef4895f4-1256-4e54-bd1d-7e2861e0f68f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:13:30.872981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:30.872981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:30.872981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:30.872981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:30.872981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:30.872981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:30.872981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:30.872981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:30.872981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:30.872981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:30.872981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:30.872981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:30.872981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:30.872981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:30.872981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:30.873999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:30.873999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:30.873999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:30.873999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:30.873999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:30.873999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:30.873999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:30.873999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:30.873999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:30.873999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:30.873999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:30.873999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:30.873999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:30.873999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:30.873999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:30.873999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:30.873999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:30.873999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:30.875044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:30.875044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:30.875044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:30.875044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:30.875044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:30.875044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:30.875044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:30.875044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:30.875044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:13:30.875044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:30.882333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 276/552] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:30.883269 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4895f4-1256-4e54-bd1d-7e2861e0f68f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:30.962731 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3c223-b292-4a0e-ae87-e69bbe7e711f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:30.994644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:31.036538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478785c4-483e-411e-be2e-e9a1ff12e4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:31.037594 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478785c4-483e-411e-be2e-e9a1ff12e4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:31.038122 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478785c4-483e-411e-be2e-e9a1ff12e4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:31.038617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478785c4-483e-411e-be2e-e9a1ff12e4b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:31.054295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4895f4-1256-4e54-bd1d-7e2861e0f68f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:31.060333 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478785c4-483e-411e-be2e-e9a1ff12e4b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:31.062026 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478785c4-483e-411e-be2e-e9a1ff12e4b4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:31.065003 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:31.090970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-478785c4-483e-411e-be2e-e9a1ff12e4b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:31.090970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:31.090970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:31.090970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:31.090970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:31.090970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:31.090970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:31.090970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:31.090970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:31.090970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:31.090970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:31.090970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:31.090970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:31.090970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:31.090970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:31.090970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:31.091922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:31.091922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:31.091922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:31.091922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:31.091922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:31.091922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:31.091922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:31.091922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:31.091922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:31.091922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:31.091922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:31.091922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:31.091922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:31.091922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:31.091922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:31.091922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:31.091922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:31.092781 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:31.092781 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:31.092781 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:31.097578 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 277/553] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:31.099661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42eae3-9c73-4f33-b413-f7365df64d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:31.100334 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42eae3-9c73-4f33-b413-f7365df64d3c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:31.100616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42eae3-9c73-4f33-b413-f7365df64d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:31.100883 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42eae3-9c73-4f33-b413-f7365df64d3c None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:31.101667 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42eae3-9c73-4f33-b413-f7365df64d3c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:31.101903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42eae3-9c73-4f33-b413-f7365df64d3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:31.102983 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478785c4-483e-411e-be2e-e9a1ff12e4b4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:31.103361 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42eae3-9c73-4f33-b413-f7365df64d3c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:31.139302 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 277/554] 10.222.0.39 () {56 vars in 1135 bytes} [Mon Aug 8 02:13:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:31.153585 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42eae3-9c73-4f33-b413-f7365df64d3c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:31.215209 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478785c4-483e-411e-be2e-e9a1ff12e4b4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:31.254286 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf767b-c1d5-443b-b6bf-f5741e1048a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:31.256703 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf767b-c1d5-443b-b6bf-f5741e1048a2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:31.257865 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf767b-c1d5-443b-b6bf-f5741e1048a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:31.259973 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf767b-c1d5-443b-b6bf-f5741e1048a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:31.261621 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf767b-c1d5-443b-b6bf-f5741e1048a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:31.266680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf767b-c1d5-443b-b6bf-f5741e1048a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:31.268087 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf767b-c1d5-443b-b6bf-f5741e1048a2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:31.289810 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 278/555] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:31.297558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a93aa3-5977-4b66-b479-cc7638b237c2 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:31.297846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a93aa3-5977-4b66-b479-cc7638b237c2 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:31.298122 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a93aa3-5977-4b66-b479-cc7638b237c2 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:31.302568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 279/556] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:31.303522 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a93aa3-5977-4b66-b479-cc7638b237c2 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:31.334398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42eae3-9c73-4f33-b413-f7365df64d3c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:31.351827 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:31.368939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:31.370463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636f06a-c2c8-45b8-8504-6f35ad6caf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:31.371295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636f06a-c2c8-45b8-8504-6f35ad6caf60 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:31.371576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636f06a-c2c8-45b8-8504-6f35ad6caf60 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:31.371864 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636f06a-c2c8-45b8-8504-6f35ad6caf60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:31.419318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a93aa3-5977-4b66-b479-cc7638b237c2 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:31.453263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636f06a-c2c8-45b8-8504-6f35ad6caf60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:31.454769 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636f06a-c2c8-45b8-8504-6f35ad6caf60 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:31.455680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd92f0-e86b-4246-9b43-28f7ee687fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:31.456318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd92f0-e86b-4246-9b43-28f7ee687fdb None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:31.456764 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd92f0-e86b-4246-9b43-28f7ee687fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:31.457044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd92f0-e86b-4246-9b43-28f7ee687fdb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:31.466159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd92f0-e86b-4246-9b43-28f7ee687fdb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:31.467571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd92f0-e86b-4246-9b43-28f7ee687fdb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:31.479110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-4edd92f0-e86b-4246-9b43-28f7ee687fdb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:13:31.479110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:31.479110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:31.479110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:31.479110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:31.479110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:31.479110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:31.479110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:31.479110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:31.479110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:31.479110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:31.479110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:31.479110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:31.479110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:31.479110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:31.479110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:31.480489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:31.480489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:31.480489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:31.480489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:31.480489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:31.480489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:31.480489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:31.480489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:31.480489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:31.480489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:31.480489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:31.480489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:31.480489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:31.480489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:31.480489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:31.480489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:31.480489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:31.480489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:31.481410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:31.481410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:31.481410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:31.481410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:31.481410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:31.481410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:31.481410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:31.481410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:31.481410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:13:31.481410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:31.481410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 280/557] 10.222.0.39 () {56 vars in 1093 bytes} [Mon Aug 8 02:13:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:31.481410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316a78c-2de0-4a6b-bc52-d7840ec1b723 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:31.482041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316a78c-2de0-4a6b-bc52-d7840ec1b723 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:31.482041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316a78c-2de0-4a6b-bc52-d7840ec1b723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:31.503130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-e316a78c-2de0-4a6b-bc52-d7840ec1b723 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:31.504503 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e316a78c-2de0-4a6b-bc52-d7840ec1b723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:31.569877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 278/558] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:31.569877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26b479-a277-42af-babb-1f37eed7fed5 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:31.569877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26b479-a277-42af-babb-1f37eed7fed5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:31.569877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26b479-a277-42af-babb-1f37eed7fed5 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:31.569877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 279/559] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:31.569877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26b479-a277-42af-babb-1f37eed7fed5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:31.590306 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 281/560] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:31.590306 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316a78c-2de0-4a6b-bc52-d7840ec1b723 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:31.595658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:31.648491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:31.682747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26b479-a277-42af-babb-1f37eed7fed5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:31.705942 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316a78c-2de0-4a6b-bc52-d7840ec1b723 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:31.719120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb40da-309f-47d2-8570-0f768354a00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:31.719668 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb40da-309f-47d2-8570-0f768354a00c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:31.719949 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb40da-309f-47d2-8570-0f768354a00c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:31.720221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb40da-309f-47d2-8570-0f768354a00c None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:31.720944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb40da-309f-47d2-8570-0f768354a00c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:31.721248 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb40da-309f-47d2-8570-0f768354a00c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:31.722722 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb40da-309f-47d2-8570-0f768354a00c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:31.740946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 280/561] 10.222.0.39 () {56 vars in 1133 bytes} [Mon Aug 8 02:13:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:31.742125 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ca4bf-aa7a-4d2c-8d3f-032fe8cdf7e5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:31.742441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ca4bf-aa7a-4d2c-8d3f-032fe8cdf7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:31.742715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ca4bf-aa7a-4d2c-8d3f-032fe8cdf7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:31.747601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21adc9-683f-489b-bd9e-6d9ba5d4b436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:31.748306 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21adc9-683f-489b-bd9e-6d9ba5d4b436 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:31.748619 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21adc9-683f-489b-bd9e-6d9ba5d4b436 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:31.748906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21adc9-683f-489b-bd9e-6d9ba5d4b436 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:31.760356 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e59ca4bf-aa7a-4d2c-8d3f-032fe8cdf7e5 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:31.760693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21adc9-683f-489b-bd9e-6d9ba5d4b436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:31.761548 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e59ca4bf-aa7a-4d2c-8d3f-032fe8cdf7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:31.762136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21adc9-683f-489b-bd9e-6d9ba5d4b436 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:31.777442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-ed21adc9-683f-489b-bd9e-6d9ba5d4b436 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:31.777442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:31.777442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:31.777442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:31.777442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:31.777442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:31.777442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:31.777442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:31.777442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:31.777442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:31.777442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:31.777442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:31.777442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:31.777442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:31.777442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:31.777442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:31.778652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:31.778652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:31.778652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:31.778652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:31.778652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:31.778652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:31.778652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:31.778652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:31.778652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:31.778652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:31.778652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:31.778652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:31.778652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:31.778652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:31.778652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:31.778652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:31.778652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:31.779509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:31.779509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:31.779509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:31.779509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 282/562] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:31.779509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21adc9-683f-489b-bd9e-6d9ba5d4b436 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:31.838347 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 281/563] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:31.838347 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ca4bf-aa7a-4d2c-8d3f-032fe8cdf7e5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:31.851109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:31.891607 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:31.896207 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21adc9-683f-489b-bd9e-6d9ba5d4b436 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:31.930257 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f1a82-f9ca-4f53-8c5a-b5bd6ab50b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:31.930903 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f1a82-f9ca-4f53-8c5a-b5bd6ab50b01 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:31.931187 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f1a82-f9ca-4f53-8c5a-b5bd6ab50b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:31.931456 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f1a82-f9ca-4f53-8c5a-b5bd6ab50b01 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:31.942821 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f1a82-f9ca-4f53-8c5a-b5bd6ab50b01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:31.948013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-456f1a82-f9ca-4f53-8c5a-b5bd6ab50b01 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:31.948013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:31.948013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:31.948013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:31.948013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:31.948013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:31.948013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:31.948013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:31.948013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:31.948013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:31.948013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:31.948013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:31.948013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:31.948013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:31.948013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:31.948013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:31.949130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:31.949130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:31.949130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:31.949130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:31.949130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:31.949130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:31.949130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:31.949130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:31.949130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:31.949130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:31.949130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:31.949130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:31.949130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:31.949130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:31.949130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:31.949130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:31.950316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:31.950316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:31.950316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:31.950316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:31.950316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:31.950316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:31.950316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:31.950316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:31.950316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:31.950316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:31.950316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:31.950316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:31.954051 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 283/564] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:31.954902 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f1a82-f9ca-4f53-8c5a-b5bd6ab50b01 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:31.957184 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ca4bf-aa7a-4d2c-8d3f-032fe8cdf7e5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:31.993694 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3e45c-21f1-44ad-81a5-e82de6f9db77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:31.994256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3e45c-21f1-44ad-81a5-e82de6f9db77 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:31.994539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3e45c-21f1-44ad-81a5-e82de6f9db77 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:31.994810 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3e45c-21f1-44ad-81a5-e82de6f9db77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.006305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3e45c-21f1-44ad-81a5-e82de6f9db77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:32.007756 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3e45c-21f1-44ad-81a5-e82de6f9db77 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:32.022285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7ec3e45c-21f1-44ad-81a5-e82de6f9db77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:32.022285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:32.022285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:32.022285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:32.022285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:32.022285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:32.022285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:32.022285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:32.022285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:32.022285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:32.022285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:32.022285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:32.022285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:32.022285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:32.022285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:32.022285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:32.023241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:32.023241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:32.023241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:32.023241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:32.023241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:32.023241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:32.023241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:32.023241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:32.023241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:32.023241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:32.023241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:32.023241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:32.023241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:32.023241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:32.023241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:32.023241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:32.023241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:32.024116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:32.024116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:32.024116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:32.024116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 282/565] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:32.024937 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3e45c-21f1-44ad-81a5-e82de6f9db77 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:32.071616 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f1a82-f9ca-4f53-8c5a-b5bd6ab50b01 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:32.090088 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:32.108239 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7f329-37d6-4bbe-8f8f-0f3e2398a467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:32.108856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7f329-37d6-4bbe-8f8f-0f3e2398a467 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.109127 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7f329-37d6-4bbe-8f8f-0f3e2398a467 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.109384 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7f329-37d6-4bbe-8f8f-0f3e2398a467 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.110054 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7f329-37d6-4bbe-8f8f-0f3e2398a467 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:32.110473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7f329-37d6-4bbe-8f8f-0f3e2398a467 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:32.111957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7f329-37d6-4bbe-8f8f-0f3e2398a467 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:32.129941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 284/566] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:32.130837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7f329-37d6-4bbe-8f8f-0f3e2398a467 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:32.133935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3e45c-21f1-44ad-81a5-e82de6f9db77 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:32.150368 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:32.170279 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb31591-060f-4698-901e-069941d171fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:32.170899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb31591-060f-4698-901e-069941d171fa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.171177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb31591-060f-4698-901e-069941d171fa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.171446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb31591-060f-4698-901e-069941d171fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.183036 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb31591-060f-4698-901e-069941d171fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:32.188522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1eb31591-060f-4698-901e-069941d171fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:32.188522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:32.188522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:32.188522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:32.188522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:32.188522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:32.188522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:32.188522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:32.188522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:32.188522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:32.188522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:32.188522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:32.188522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:32.188522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:32.188522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:32.188522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:32.189621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:32.189621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:32.189621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:32.189621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:32.189621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:32.189621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:32.189621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:32.189621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:32.189621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:32.189621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:32.189621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:32.189621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:32.189621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:32.189621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:32.189621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:32.189621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:32.190950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:32.190950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:32.190950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:32.190950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:32.190950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:32.190950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:32.190950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:32.190950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:32.190950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:32.190950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:32.190950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:32.190950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:32.190950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 283/567] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:32.190950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb31591-060f-4698-901e-069941d171fa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:32.244417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7f329-37d6-4bbe-8f8f-0f3e2398a467 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:32.280205 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f976420-2bba-4f2e-ae24-2aa6be067e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:32.280811 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f976420-2bba-4f2e-ae24-2aa6be067e00 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.281098 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f976420-2bba-4f2e-ae24-2aa6be067e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.281374 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f976420-2bba-4f2e-ae24-2aa6be067e00 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.282031 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f976420-2bba-4f2e-ae24-2aa6be067e00 None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:32.282359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f976420-2bba-4f2e-ae24-2aa6be067e00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:32.283988 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f976420-2bba-4f2e-ae24-2aa6be067e00 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:32.292846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb31591-060f-4698-901e-069941d171fa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:32.297617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 285/568] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:13:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:32.298505 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f976420-2bba-4f2e-ae24-2aa6be067e00 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:32.310301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:32.328388 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c982110-c0f6-44b6-a2b9-1272dd6b148e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:32.329080 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c982110-c0f6-44b6-a2b9-1272dd6b148e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.329441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c982110-c0f6-44b6-a2b9-1272dd6b148e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.329789 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c982110-c0f6-44b6-a2b9-1272dd6b148e None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.330533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c982110-c0f6-44b6-a2b9-1272dd6b148e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:32.330913 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c982110-c0f6-44b6-a2b9-1272dd6b148e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:32.332347 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c982110-c0f6-44b6-a2b9-1272dd6b148e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:32.350028 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 284/569] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:32.350028 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c982110-c0f6-44b6-a2b9-1272dd6b148e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:32.392929 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:32.415197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f976420-2bba-4f2e-ae24-2aa6be067e00 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:32.449477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912ab72-e985-4e93-908d-e2a84043113f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:32.450140 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912ab72-e985-4e93-908d-e2a84043113f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.450273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c982110-c0f6-44b6-a2b9-1272dd6b148e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:32.450457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912ab72-e985-4e93-908d-e2a84043113f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.450734 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912ab72-e985-4e93-908d-e2a84043113f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.457344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912ab72-e985-4e93-908d-e2a84043113f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:32.459353 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912ab72-e985-4e93-908d-e2a84043113f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:32.471139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-0912ab72-e985-4e93-908d-e2a84043113f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:32.471139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:32.471139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:32.471139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:32.471139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:32.471139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:32.471139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:32.471139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:32.471139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:32.471139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:32.471139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:32.471139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:32.471139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:32.471139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:32.471139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:32.472099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:32.472099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:32.472099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:32.472099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:32.472099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:32.472099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:32.472099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:32.472099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:32.472099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:32.472099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:32.472099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:32.472099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:32.472099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:32.472099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:32.472099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:32.472099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:32.472099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:32.472982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:32.472982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:32.472982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:32.472982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:32.472982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:32.472982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:32.472982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:32.472982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:32.472982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 286/570] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:32.472982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912ab72-e985-4e93-908d-e2a84043113f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:32.489172 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98ba15-9224-4cfd-9606-224dbae8dcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:32.489783 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98ba15-9224-4cfd-9606-224dbae8dcf2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.490064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98ba15-9224-4cfd-9606-224dbae8dcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.490363 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98ba15-9224-4cfd-9606-224dbae8dcf2 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.491032 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98ba15-9224-4cfd-9606-224dbae8dcf2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:32.491338 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98ba15-9224-4cfd-9606-224dbae8dcf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:32.492716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98ba15-9224-4cfd-9606-224dbae8dcf2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:32.500884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 285/571] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:13:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:32.501599 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98ba15-9224-4cfd-9606-224dbae8dcf2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:32.545749 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:32.574207 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912ab72-e985-4e93-908d-e2a84043113f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:32.604009 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98ba15-9224-4cfd-9606-224dbae8dcf2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:32.607366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c0055-ec63-418f-bfb8-55548ee32f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:32.607981 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c0055-ec63-418f-bfb8-55548ee32f18 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.608260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c0055-ec63-418f-bfb8-55548ee32f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.608531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c0055-ec63-418f-bfb8-55548ee32f18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.616900 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:32.619409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c0055-ec63-418f-bfb8-55548ee32f18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:32.621344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c0055-ec63-418f-bfb8-55548ee32f18 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:32.635297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-2a8c0055-ec63-418f-bfb8-55548ee32f18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:32.635297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:32.635297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:32.635297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:32.635297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:32.635297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:32.635297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:32.635297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:32.635297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:32.635297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:32.635297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:32.635297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:32.635297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:32.635297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:32.635297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:32.635297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:32.636284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:32.636284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:32.636284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:32.636284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:32.636284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:32.636284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:32.636284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:32.636284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:32.636284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:32.636284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:32.636284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:32.636284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:32.636284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:32.636284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:32.636284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:32.636284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:32.636284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:32.637872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:32.637872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:32.637872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:32.637872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 287/572] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:32.637872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd727b-3b92-4612-acc0-f311aac834c5 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.638226 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd727b-3b92-4612-acc0-f311aac834c5 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.638421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd727b-3b92-4612-acc0-f311aac834c5 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.639630 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 288/573] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:32.640315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd727b-3b92-4612-acc0-f311aac834c5 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:32.642925 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0877d0-6821-4906-8cf0-a328cbd6da71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:32.643635 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0877d0-6821-4906-8cf0-a328cbd6da71 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.643918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0877d0-6821-4906-8cf0-a328cbd6da71 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.644193 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0877d0-6821-4906-8cf0-a328cbd6da71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.651586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0877d0-6821-4906-8cf0-a328cbd6da71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:32.653035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0877d0-6821-4906-8cf0-a328cbd6da71 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:32.665260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-8c0877d0-6821-4906-8cf0-a328cbd6da71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:32.665260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:32.665260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:32.665260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:32.665260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:32.665260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:32.665260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:32.665260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:32.665260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:32.665260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:32.665260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:32.665260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:32.665260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:32.665260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:32.665260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:32.666136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:32.666136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:32.666136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:32.666136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:32.666136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:32.666136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:32.666136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:32.666136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:32.666136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:32.666136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:32.666136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:32.666136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:32.666136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:32.666136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:32.666136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:32.666136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:32.666136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:32.667053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:32.667053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:32.667053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:32.667053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:32.667053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:32.667053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:32.667053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:32.667053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:32.667053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 286/574] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:32.667537 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0877d0-6821-4906-8cf0-a328cbd6da71 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:32.748516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd727b-3b92-4612-acc0-f311aac834c5 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:32.764578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:32.767605 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0877d0-6821-4906-8cf0-a328cbd6da71 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:32.781437 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9c292-9637-4e2d-9855-a2bcffa32875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:32.782040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9c292-9637-4e2d-9855-a2bcffa32875 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.782349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9c292-9637-4e2d-9855-a2bcffa32875 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.782623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9c292-9637-4e2d-9855-a2bcffa32875 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.783310 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff9c292-9637-4e2d-9855-a2bcffa32875 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:32.783619 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff9c292-9637-4e2d-9855-a2bcffa32875 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:32.785043 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff9c292-9637-4e2d-9855-a2bcffa32875 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:32.797715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 289/575] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:32.798549 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9c292-9637-4e2d-9855-a2bcffa32875 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:32.806898 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0bcde-747f-4b03-8f16-4f5438661911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:32.807517 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0bcde-747f-4b03-8f16-4f5438661911 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.807795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0bcde-747f-4b03-8f16-4f5438661911 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.808064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0bcde-747f-4b03-8f16-4f5438661911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.819155 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0bcde-747f-4b03-8f16-4f5438661911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:32.820607 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0bcde-747f-4b03-8f16-4f5438661911 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:32.834686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-45d0bcde-747f-4b03-8f16-4f5438661911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:32.834686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:32.834686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:32.834686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:32.834686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:32.834686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:32.834686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:32.834686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:32.834686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:32.834686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:32.834686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:32.834686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:32.834686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:32.834686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:32.834686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:32.834686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:32.835753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:32.835753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:32.835753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:32.835753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:32.835753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:32.835753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:32.835753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:32.835753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:32.835753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:32.835753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:32.835753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:32.835753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:32.835753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:32.835753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:32.835753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:32.835753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:32.835753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:32.836637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:32.836637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:32.836637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:32.836637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 287/576] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:32.836864 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29bacc-a1db-4e8d-a833-02f16f55ed9a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.837235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29bacc-a1db-4e8d-a833-02f16f55ed9a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.837397 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29bacc-a1db-4e8d-a833-02f16f55ed9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.853164 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:32.854263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-df29bacc-a1db-4e8d-a833-02f16f55ed9a None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:32.855473 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-df29bacc-a1db-4e8d-a833-02f16f55ed9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:32.900963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9c292-9637-4e2d-9855-a2bcffa32875 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:32.921277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 288/577] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:32.922177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29bacc-a1db-4e8d-a833-02f16f55ed9a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:32.933890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd0712-d41d-47fb-8b7f-2d279b5b518c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:32.934516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd0712-d41d-47fb-8b7f-2d279b5b518c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.935050 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd0712-d41d-47fb-8b7f-2d279b5b518c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.935319 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd0712-d41d-47fb-8b7f-2d279b5b518c None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.935958 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd0712-d41d-47fb-8b7f-2d279b5b518c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:32.936260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd0712-d41d-47fb-8b7f-2d279b5b518c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:32.937606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd0712-d41d-47fb-8b7f-2d279b5b518c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:32.949208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 290/578] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:32.950004 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd0712-d41d-47fb-8b7f-2d279b5b518c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.004275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:33.026358 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29bacc-a1db-4e8d-a833-02f16f55ed9a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:33.053265 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd0712-d41d-47fb-8b7f-2d279b5b518c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:33.061584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa681ee-7016-403d-adf9-12b99f494268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:33.062179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa681ee-7016-403d-adf9-12b99f494268 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.062482 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa681ee-7016-403d-adf9-12b99f494268 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.062748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa681ee-7016-403d-adf9-12b99f494268 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.063444 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa681ee-7016-403d-adf9-12b99f494268 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:33.063724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa681ee-7016-403d-adf9-12b99f494268 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:33.065158 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa681ee-7016-403d-adf9-12b99f494268 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:33.065707 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:33.077523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 289/579] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:33.078346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa681ee-7016-403d-adf9-12b99f494268 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.088656 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960892e4-db16-49c2-b00b-0a943d46ca24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:33.089259 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960892e4-db16-49c2-b00b-0a943d46ca24 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.089537 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960892e4-db16-49c2-b00b-0a943d46ca24 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.089809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960892e4-db16-49c2-b00b-0a943d46ca24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.090561 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960892e4-db16-49c2-b00b-0a943d46ca24 None admin] RBAC: Adding query filter params (role.id=838b081e674f4c0ab4a33bcf8e614dc5, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=ac60b4d621234225830c94edfac400e2) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:33.090876 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960892e4-db16-49c2-b00b-0a943d46ca24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:33.092284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960892e4-db16-49c2-b00b-0a943d46ca24 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:33.097708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 291/580] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:32 2022] GET /identity/v3/role_assignments?user.id=ac60b4d621234225830c94edfac400e2&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=838b081e674f4c0ab4a33bcf8e614dc5 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:33.098581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960892e4-db16-49c2-b00b-0a943d46ca24 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.202463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa681ee-7016-403d-adf9-12b99f494268 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:33.217228 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960892e4-db16-49c2-b00b-0a943d46ca24 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:33.239575 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42792a-4714-4378-a5b6-922bc1df9920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:33.240187 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42792a-4714-4378-a5b6-922bc1df9920 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.240465 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42792a-4714-4378-a5b6-922bc1df9920 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.240744 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42792a-4714-4378-a5b6-922bc1df9920 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.241453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42792a-4714-4378-a5b6-922bc1df9920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:33.241764 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42792a-4714-4378-a5b6-922bc1df9920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:33.243202 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42792a-4714-4378-a5b6-922bc1df9920 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:33.261189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 290/581] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:33.261189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42792a-4714-4378-a5b6-922bc1df9920 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.261516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d6f18-34e3-4274-8ba0-76250c149548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:33.261516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d6f18-34e3-4274-8ba0-76250c149548 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.261516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d6f18-34e3-4274-8ba0-76250c149548 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.261516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d6f18-34e3-4274-8ba0-76250c149548 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.263014 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:33.274324 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d6f18-34e3-4274-8ba0-76250c149548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:33.279798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-959d6f18-34e3-4274-8ba0-76250c149548 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:33.279798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:33.279798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:33.279798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:33.279798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:33.279798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:33.279798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:33.279798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:33.279798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:33.279798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:33.279798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:33.279798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:33.279798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:33.279798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:33.279798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:33.279798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:33.280708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:33.280708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:33.280708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:33.280708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:33.280708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:33.280708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:33.280708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:33.280708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:33.280708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:33.280708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:33.280708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:33.280708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:33.280708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:33.280708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:33.280708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:33.280708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:33.281617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:33.281617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:33.281617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:33.281617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:33.281617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:33.281617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:33.281617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:33.281617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:33.281617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:33.281617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:33.281617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:33.281617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:33.283014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 292/582] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:33.283248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d6f18-34e3-4274-8ba0-76250c149548 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.324032 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:33.371083 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42792a-4714-4378-a5b6-922bc1df9920 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:33.395461 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d6f18-34e3-4274-8ba0-76250c149548 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:33.412681 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf9ad1-5706-4f13-b44d-4a02740e55ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:33.413304 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf9ad1-5706-4f13-b44d-4a02740e55ce None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.413583 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf9ad1-5706-4f13-b44d-4a02740e55ce None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.413851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf9ad1-5706-4f13-b44d-4a02740e55ce None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.423523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf9ad1-5706-4f13-b44d-4a02740e55ce None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:33.425116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf9ad1-5706-4f13-b44d-4a02740e55ce None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:33.436279 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc289904-a9d5-4ef5-8f88-6abcd338db80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:33.436927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc289904-a9d5-4ef5-8f88-6abcd338db80 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.437202 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc289904-a9d5-4ef5-8f88-6abcd338db80 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.437472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc289904-a9d5-4ef5-8f88-6abcd338db80 None admin] PATH_INFO: `/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/c4238566a9c248ffbfa314f47f02670c/roles/838b081e674f4c0ab4a33bcf8e614dc5` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.438773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-44cf9ad1-5706-4f13-b44d-4a02740e55ce None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:13:33.438773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:33.438773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:33.438773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:33.438773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:33.438773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:33.438773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:33.438773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:33.438773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:33.438773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:33.438773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:33.438773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:33.438773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:33.438773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:33.438773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:33.438773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:33.439652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:33.439652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:33.439652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:33.439652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:33.439652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:33.439652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:33.439652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:33.439652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:33.439652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:33.439652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:33.439652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:33.439652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:33.439652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:33.439652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:33.439652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:33.439652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:33.439652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:33.439652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:33.440602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:33.440602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:33.440602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:33.440602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:33.440602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:33.440602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:33.440602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:33.440602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:33.440602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:13:33.440602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:33.440602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 291/583] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:33.441192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf9ad1-5706-4f13-b44d-4a02740e55ce None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.461434 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc289904-a9d5-4ef5-8f88-6abcd338db80 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d474b61f1fe4f17aacb7a3665315c79, user_id=c4238566a9c248ffbfa314f47f02670c, role_id=838b081e674f4c0ab4a33bcf8e614dc5)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:33.462911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc289904-a9d5-4ef5-8f88-6abcd338db80 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:33.484296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 293/584] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:13:33 2022] PUT /identity/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/c4238566a9c248ffbfa314f47f02670c/roles/838b081e674f4c0ab4a33bcf8e614dc5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:33.484296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc289904-a9d5-4ef5-8f88-6abcd338db80 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.528707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:33.570551 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf9ad1-5706-4f13-b44d-4a02740e55ce None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:33.587766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:33.608903 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc289904-a9d5-4ef5-8f88-6abcd338db80 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:33.612793 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903642d-0982-46f1-93d5-c41a05e051a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:33.613635 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903642d-0982-46f1-93d5-c41a05e051a1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.614679 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903642d-0982-46f1-93d5-c41a05e051a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.615098 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903642d-0982-46f1-93d5-c41a05e051a1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.624012 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903642d-0982-46f1-93d5-c41a05e051a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:33.625773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903642d-0982-46f1-93d5-c41a05e051a1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:33.640120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c903642d-0982-46f1-93d5-c41a05e051a1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 02:13:33.640120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:33.640120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:33.640120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:33.640120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:33.640120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:33.640120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:33.640120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:33.640120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:33.640120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:33.640120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:33.640120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:33.640120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:33.640120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:33.640120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:33.640120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:33.641229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:33.641229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:33.641229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:33.641229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:33.641229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:33.641229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:33.641229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:33.641229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:33.641229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:33.641229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:33.641229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:33.641229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:33.641229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:33.641229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:33.641229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:33.641229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:33.641229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:33.641229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:33.642189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:33.642189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:33.642189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:33.642189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:33.642189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:33.642189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:33.642189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:33.642189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:33.642189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 02:13:33.642189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:33.644795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 292/585] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:33.646062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903642d-0982-46f1-93d5-c41a05e051a1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.661719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0519b2-70a3-4ff8-a0a6-b5e5071d26af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:33.662489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0519b2-70a3-4ff8-a0a6-b5e5071d26af None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.662754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0519b2-70a3-4ff8-a0a6-b5e5071d26af None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.663028 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0519b2-70a3-4ff8-a0a6-b5e5071d26af None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.663698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0519b2-70a3-4ff8-a0a6-b5e5071d26af None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:33.664107 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0519b2-70a3-4ff8-a0a6-b5e5071d26af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:33.665467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0519b2-70a3-4ff8-a0a6-b5e5071d26af None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:33.677923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 294/586] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:13:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:33.678930 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0519b2-70a3-4ff8-a0a6-b5e5071d26af None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.770792 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903642d-0982-46f1-93d5-c41a05e051a1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:33.806845 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:33.807822 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0519b2-70a3-4ff8-a0a6-b5e5071d26af None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:33.823649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df961296-1c1e-402d-8d51-850b7254e185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:33.824185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df961296-1c1e-402d-8d51-850b7254e185 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.824467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df961296-1c1e-402d-8d51-850b7254e185 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.824741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df961296-1c1e-402d-8d51-850b7254e185 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.825760 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df961296-1c1e-402d-8d51-850b7254e185 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:33.826069 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df961296-1c1e-402d-8d51-850b7254e185 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:33.828188 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df961296-1c1e-402d-8d51-850b7254e185 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:33.851013 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 293/587] 10.222.0.39 () {56 vars in 1137 bytes} [Mon Aug 8 02:13:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:33.851197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046c855-8454-4699-88f9-bf5a72d1c485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:33.851911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046c855-8454-4699-88f9-bf5a72d1c485 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.852184 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046c855-8454-4699-88f9-bf5a72d1c485 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.852458 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046c855-8454-4699-88f9-bf5a72d1c485 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.853034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df961296-1c1e-402d-8d51-850b7254e185 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.853203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046c855-8454-4699-88f9-bf5a72d1c485 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:33.853514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046c855-8454-4699-88f9-bf5a72d1c485 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:33.855030 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046c855-8454-4699-88f9-bf5a72d1c485 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:33.887276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 295/588] 10.222.0.39 () {56 vars in 1137 bytes} [Mon Aug 8 02:13:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:33.887276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046c855-8454-4699-88f9-bf5a72d1c485 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.894789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:33.967089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df961296-1c1e-402d-8d51-850b7254e185 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:34.003652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48696100-d26b-482c-9096-33b400ebf033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:34.004780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48696100-d26b-482c-9096-33b400ebf033 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:34.005268 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48696100-d26b-482c-9096-33b400ebf033 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:34.005785 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48696100-d26b-482c-9096-33b400ebf033 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:34.016942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48696100-d26b-482c-9096-33b400ebf033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:34.017494 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046c855-8454-4699-88f9-bf5a72d1c485 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:34.019008 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48696100-d26b-482c-9096-33b400ebf033 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:34.034860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-48696100-d26b-482c-9096-33b400ebf033 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:34.034860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:34.034860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:34.034860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:34.034860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:34.034860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:34.034860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:34.034860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:34.034860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:34.034860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:34.034860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:34.034860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:34.034860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:34.034860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:34.034860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:34.034860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:34.036434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:34.036434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:34.036434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:34.036434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:34.036434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:34.036434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:34.036434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:34.036434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:34.036434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:34.036434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:34.036434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:34.036434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:34.036434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:34.036434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:34.036434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:34.036434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:34.036434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:34.037374 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:34.037374 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:34.037374 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:34.044010 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 294/589] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:34.045330 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48696100-d26b-482c-9096-33b400ebf033 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:34.054012 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d52e60-65dd-4f85-8ed8-c28263251b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:34.054697 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d52e60-65dd-4f85-8ed8-c28263251b4b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:34.054978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d52e60-65dd-4f85-8ed8-c28263251b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:34.055255 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d52e60-65dd-4f85-8ed8-c28263251b4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:34.066788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d52e60-65dd-4f85-8ed8-c28263251b4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:34.068693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d52e60-65dd-4f85-8ed8-c28263251b4b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:34.081508 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:34.087285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-a5d52e60-65dd-4f85-8ed8-c28263251b4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:34.087285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:34.087285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:34.087285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:34.087285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:34.087285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:34.087285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:34.087285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:34.087285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:34.087285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:34.087285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:34.087285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:34.087285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:34.087285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:34.087285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:34.087285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:34.088190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:34.088190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:34.088190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:34.088190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:34.088190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:34.088190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:34.088190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:34.088190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:34.088190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:34.088190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:34.088190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:34.088190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:34.088190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:34.088190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:34.088190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:34.088190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:34.088190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:34.089414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:34.089414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:34.089414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:34.089414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 296/590] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:34.089414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d52e60-65dd-4f85-8ed8-c28263251b4b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:34.153162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:34.160634 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48696100-d26b-482c-9096-33b400ebf033 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:34.198301 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d52e60-65dd-4f85-8ed8-c28263251b4b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:34.199129 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962747e5-b525-4463-b12e-8aa03f69f509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:34.200111 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962747e5-b525-4463-b12e-8aa03f69f509 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:34.200571 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962747e5-b525-4463-b12e-8aa03f69f509 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:34.201037 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962747e5-b525-4463-b12e-8aa03f69f509 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:34.214499 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962747e5-b525-4463-b12e-8aa03f69f509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:34.216284 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962747e5-b525-4463-b12e-8aa03f69f509 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:34.232359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-962747e5-b525-4463-b12e-8aa03f69f509 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:34.232359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:34.232359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:34.232359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:34.232359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:34.232359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:34.232359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:34.232359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:34.232359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:34.232359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:34.232359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:34.232359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:34.232359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:34.232359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:34.232359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:34.232359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:34.233528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:34.233528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:34.233528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:34.233528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:34.233528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:34.233528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:34.233528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:34.233528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:34.233528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:34.233528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:34.233528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:34.233528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:34.233528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:34.233528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:34.233528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:34.233528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:34.233528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:34.234661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:34.234661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:34.234661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:34.234829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 295/591] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:34.238284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314621b-4050-4640-8fe1-5a826ac2dc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:34.238956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962747e5-b525-4463-b12e-8aa03f69f509 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:34.239366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314621b-4050-4640-8fe1-5a826ac2dc97 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:34.239849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314621b-4050-4640-8fe1-5a826ac2dc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:34.240318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314621b-4050-4640-8fe1-5a826ac2dc97 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:34.241182 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314621b-4050-4640-8fe1-5a826ac2dc97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:34.241710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314621b-4050-4640-8fe1-5a826ac2dc97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:34.243343 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314621b-4050-4640-8fe1-5a826ac2dc97 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:34.256663 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 297/592] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:34.257826 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314621b-4050-4640-8fe1-5a826ac2dc97 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:34.339021 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:34.360025 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962747e5-b525-4463-b12e-8aa03f69f509 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:34.362021 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314621b-4050-4640-8fe1-5a826ac2dc97 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:34.380462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:34.399022 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9ac5d-f3c7-4175-a7b0-e46be887ae14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:34.399501 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c7e80-a774-448e-8d5f-5c6a65123178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:34.399889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c7e80-a774-448e-8d5f-5c6a65123178 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:34.400288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c7e80-a774-448e-8d5f-5c6a65123178 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:34.400563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9ac5d-f3c7-4175-a7b0-e46be887ae14 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:34.400741 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c7e80-a774-448e-8d5f-5c6a65123178 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:34.401131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9ac5d-f3c7-4175-a7b0-e46be887ae14 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:34.401569 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c7e80-a774-448e-8d5f-5c6a65123178 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:34.401762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9ac5d-f3c7-4175-a7b0-e46be887ae14 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:34.402057 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c7e80-a774-448e-8d5f-5c6a65123178 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:34.402710 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9ac5d-f3c7-4175-a7b0-e46be887ae14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:34.403147 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9ac5d-f3c7-4175-a7b0-e46be887ae14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:34.403730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c7e80-a774-448e-8d5f-5c6a65123178 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:34.404790 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9ac5d-f3c7-4175-a7b0-e46be887ae14 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:34.419921 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 298/593] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:34.419921 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c7e80-a774-448e-8d5f-5c6a65123178 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:34.428142 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 296/594] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:34.429307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9ac5d-f3c7-4175-a7b0-e46be887ae14 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:34.521645 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c7e80-a774-448e-8d5f-5c6a65123178 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:34.543183 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9ac5d-f3c7-4175-a7b0-e46be887ae14 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:34.556425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af70043-e22c-4e65-b582-014f86dddd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:34.557137 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af70043-e22c-4e65-b582-014f86dddd1c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:34.557514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af70043-e22c-4e65-b582-014f86dddd1c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:34.557889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af70043-e22c-4e65-b582-014f86dddd1c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:34.566572 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af70043-e22c-4e65-b582-014f86dddd1c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:34.568698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af70043-e22c-4e65-b582-014f86dddd1c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:34.580671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-8af70043-e22c-4e65-b582-014f86dddd1c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 02:13:34.580671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:34.580671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:34.580671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:34.580671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:34.580671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:34.580671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:34.580671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:34.580671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:34.580671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:34.580671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:34.580671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:34.580671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:34.580671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:34.580671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:34.580671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:34.581680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:34.581680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:34.581680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:34.581680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:34.581680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:34.581680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:34.581680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:34.581680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:34.581680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:34.581680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:34.581680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:34.581680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:34.581680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:34.581680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:34.581680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:34.581680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:34.581680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:34.581680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:34.583271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:34.583271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:34.583271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:34.583271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:34.583271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:34.583271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:34.583271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:34.583271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:34.583271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 02:13:34.583271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:34.584375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36649308-4d62-4dda-89f3-fb06414609ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:34.585238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36649308-4d62-4dda-89f3-fb06414609ba None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:34.585670 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36649308-4d62-4dda-89f3-fb06414609ba None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:34.586121 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36649308-4d62-4dda-89f3-fb06414609ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:34.587576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 299/595] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:34.588622 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af70043-e22c-4e65-b582-014f86dddd1c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:34.595548 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36649308-4d62-4dda-89f3-fb06414609ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:34.597661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36649308-4d62-4dda-89f3-fb06414609ba None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:34.611371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-36649308-4d62-4dda-89f3-fb06414609ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:34.611371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:34.611371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:34.611371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:34.611371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:34.611371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:34.611371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:34.611371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:34.611371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:34.611371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:34.611371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:34.611371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:34.611371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:34.611371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:34.611371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:34.612308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:34.612308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:34.612308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:34.612308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:34.612308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:34.612308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:34.612308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:34.612308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:34.612308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:34.612308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:34.612308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:34.612308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:34.612308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:34.612308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:34.612308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:34.612308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:34.612308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:34.613174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:34.613174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:34.613174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:34.613174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:34.613174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:34.613174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:34.613174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:34.613174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:34.619900 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 297/596] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:34.620970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36649308-4d62-4dda-89f3-fb06414609ba None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:34.625029 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:34.632803 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:34.703742 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af70043-e22c-4e65-b582-014f86dddd1c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:34.735657 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36649308-4d62-4dda-89f3-fb06414609ba None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:34.741634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c104fe-298e-4e4a-a993-91d4a89f93c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:34.743575 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c104fe-298e-4e4a-a993-91d4a89f93c3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:34.743877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c104fe-298e-4e4a-a993-91d4a89f93c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:34.744153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c104fe-298e-4e4a-a993-91d4a89f93c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:34.752038 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c104fe-298e-4e4a-a993-91d4a89f93c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:34.753769 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c104fe-298e-4e4a-a993-91d4a89f93c3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:34.767253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-70c104fe-298e-4e4a-a993-91d4a89f93c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:34.767253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:34.767253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:34.767253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:34.767253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:34.767253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:34.767253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:34.767253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:34.767253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:34.767253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:34.767253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:34.767253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:34.767253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:34.767253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:34.767253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:34.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:34.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:34.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:34.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:34.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:34.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:34.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:34.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:34.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:34.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:34.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:34.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:34.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:34.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:34.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:34.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:34.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:34.769285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:34.769285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:34.769285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:34.769285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:34.769285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:34.769285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:34.769285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:34.769285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:34.774329 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 300/597] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:34.775335 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c104fe-298e-4e4a-a993-91d4a89f93c3 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:34.779721 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4adfc-6f8b-4700-9f3b-f800a53f9cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:34.780422 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4adfc-6f8b-4700-9f3b-f800a53f9cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:34.780772 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4adfc-6f8b-4700-9f3b-f800a53f9cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:34.781108 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4adfc-6f8b-4700-9f3b-f800a53f9cb9 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:34.781915 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd4adfc-6f8b-4700-9f3b-f800a53f9cb9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:34.782300 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd4adfc-6f8b-4700-9f3b-f800a53f9cb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:34.783851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd4adfc-6f8b-4700-9f3b-f800a53f9cb9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:34.806982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 298/598] 10.222.0.39 () {56 vars in 1139 bytes} [Mon Aug 8 02:13:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:34.808176 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4adfc-6f8b-4700-9f3b-f800a53f9cb9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:34.883712 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:34.893552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:34.898380 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c104fe-298e-4e4a-a993-91d4a89f93c3 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:34.927515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4adfc-6f8b-4700-9f3b-f800a53f9cb9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:34.936930 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877a2f8-e674-4744-b4f7-25d117a3ccfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:34.937840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877a2f8-e674-4744-b4f7-25d117a3ccfa None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:34.938439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877a2f8-e674-4744-b4f7-25d117a3ccfa None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:34.938908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877a2f8-e674-4744-b4f7-25d117a3ccfa None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:34.939785 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877a2f8-e674-4744-b4f7-25d117a3ccfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:34.940287 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877a2f8-e674-4744-b4f7-25d117a3ccfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:34.942695 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877a2f8-e674-4744-b4f7-25d117a3ccfa None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:34.960816 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 301/599] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:34.962506 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877a2f8-e674-4744-b4f7-25d117a3ccfa None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:34.966779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faba3b1-be35-4263-8f32-e196b57d7276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:34.968173 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faba3b1-be35-4263-8f32-e196b57d7276 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:34.968308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faba3b1-be35-4263-8f32-e196b57d7276 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:34.968511 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faba3b1-be35-4263-8f32-e196b57d7276 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:34.969429 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faba3b1-be35-4263-8f32-e196b57d7276 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:34.969612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faba3b1-be35-4263-8f32-e196b57d7276 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:34.971111 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faba3b1-be35-4263-8f32-e196b57d7276 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:34.983771 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 299/600] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:34.985003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faba3b1-be35-4263-8f32-e196b57d7276 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:35.081557 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877a2f8-e674-4744-b4f7-25d117a3ccfa None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:35.087021 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faba3b1-be35-4263-8f32-e196b57d7276 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:35.110231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:35.123428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e50307-c13f-4b92-b8ff-7afce3cd2209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:35.123764 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13d39e-4302-4a09-98b7-b33d7beebb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:35.124242 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e50307-c13f-4b92-b8ff-7afce3cd2209 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:35.124400 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13d39e-4302-4a09-98b7-b33d7beebb61 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:35.124674 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13d39e-4302-4a09-98b7-b33d7beebb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:35.124941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13d39e-4302-4a09-98b7-b33d7beebb61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:35.125332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e50307-c13f-4b92-b8ff-7afce3cd2209 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:35.125774 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e50307-c13f-4b92-b8ff-7afce3cd2209 None admin] PATH_INFO: `/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/7776a6a1faf34b369e399d83bb3e18c7/roles/838b081e674f4c0ab4a33bcf8e614dc5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:35.135934 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13d39e-4302-4a09-98b7-b33d7beebb61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:35.137306 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13d39e-4302-4a09-98b7-b33d7beebb61 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:35.141302 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e50307-c13f-4b92-b8ff-7afce3cd2209 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d474b61f1fe4f17aacb7a3665315c79, user_id=7776a6a1faf34b369e399d83bb3e18c7, role_id=838b081e674f4c0ab4a33bcf8e614dc5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:35.142952 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e50307-c13f-4b92-b8ff-7afce3cd2209 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:35.152421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-5b13d39e-4302-4a09-98b7-b33d7beebb61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:35.152421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:35.152421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:35.152421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:35.152421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:35.152421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:35.152421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:35.152421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:35.152421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:35.152421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:35.152421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:35.152421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:35.152421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:35.152421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:35.152421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:35.152421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:35.153369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:35.153369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:35.153369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:35.153369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:35.153369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:35.153369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:35.153369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:35.153369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:35.153369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:35.153369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:35.153369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:35.153369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:35.153369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:35.153369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:35.153369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:35.153369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:35.153369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:35.154221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:35.154221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:35.154221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:35.154221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 302/601] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:35.154493 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13d39e-4302-4a09-98b7-b33d7beebb61 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:35.160378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 300/602] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:13:34 2022] PUT /identity/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/7776a6a1faf34b369e399d83bb3e18c7/roles/838b081e674f4c0ab4a33bcf8e614dc5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:35.162335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190f193-b1d4-4f05-91f0-41f668b20e50 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:35.162791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190f193-b1d4-4f05-91f0-41f668b20e50 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:35.163254 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190f193-b1d4-4f05-91f0-41f668b20e50 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:35.164950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 301/603] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:35.165893 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190f193-b1d4-4f05-91f0-41f668b20e50 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:35.183074 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:35.303488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13d39e-4302-4a09-98b7-b33d7beebb61 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:35.316346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190f193-b1d4-4f05-91f0-41f668b20e50 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:35.356549 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01bb085-898b-435b-9544-a92a6f731147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:35.357211 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bb085-898b-435b-9544-a92a6f731147 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:35.357494 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bb085-898b-435b-9544-a92a6f731147 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:35.357776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bb085-898b-435b-9544-a92a6f731147 None admin] PATH_INFO: `/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/2812e59d0d9948879a465fe3b5b421e1/roles/838b081e674f4c0ab4a33bcf8e614dc5` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:35.358984 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54935c75-3548-4ddd-8688-ae2bccb62ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:35.359566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54935c75-3548-4ddd-8688-ae2bccb62ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:35.359840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54935c75-3548-4ddd-8688-ae2bccb62ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:35.360111 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54935c75-3548-4ddd-8688-ae2bccb62ed1 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:35.360748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54935c75-3548-4ddd-8688-ae2bccb62ed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:35.361052 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54935c75-3548-4ddd-8688-ae2bccb62ed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:35.364740 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54935c75-3548-4ddd-8688-ae2bccb62ed1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:35.376886 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01bb085-898b-435b-9544-a92a6f731147 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d474b61f1fe4f17aacb7a3665315c79, user_id=2812e59d0d9948879a465fe3b5b421e1, role_id=838b081e674f4c0ab4a33bcf8e614dc5)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:35.378353 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01bb085-898b-435b-9544-a92a6f731147 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:35.379215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 302/604] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:35.381056 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34feab47-5b03-4e2b-a2ea-de7bd9337fe9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:35.381591 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34feab47-5b03-4e2b-a2ea-de7bd9337fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:35.382064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34feab47-5b03-4e2b-a2ea-de7bd9337fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:35.410372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 303/605] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:13:35 2022] PUT /identity/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/2812e59d0d9948879a465fe3b5b421e1/roles/838b081e674f4c0ab4a33bcf8e614dc5 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:35.410372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01bb085-898b-435b-9544-a92a6f731147 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:35.412966 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-34feab47-5b03-4e2b-a2ea-de7bd9337fe9 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:35.414746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-34feab47-5b03-4e2b-a2ea-de7bd9337fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:35.475345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:35.485551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:35.557201 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01bb085-898b-435b-9544-a92a6f731147 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:35.561051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 303/606] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:35.587873 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34feab47-5b03-4e2b-a2ea-de7bd9337fe9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:35.624092 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9deed-b1b4-4fdd-b519-475eada3098c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:35.624867 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9deed-b1b4-4fdd-b519-475eada3098c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:35.625465 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9deed-b1b4-4fdd-b519-475eada3098c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:35.625748 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9deed-b1b4-4fdd-b519-475eada3098c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:35.635579 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9deed-b1b4-4fdd-b519-475eada3098c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:35.637036 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9deed-b1b4-4fdd-b519-475eada3098c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:35.648737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-dfb9deed-b1b4-4fdd-b519-475eada3098c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:35.648737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:35.648737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:35.648737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:35.648737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:35.648737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:35.648737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:35.648737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:35.648737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:35.648737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:35.648737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:35.648737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:35.648737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:35.648737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:35.648737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:35.649796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:35.649796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:35.649796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:35.649796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:35.649796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:35.649796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:35.649796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:35.649796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:35.649796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:35.649796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:35.649796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:35.649796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:35.649796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:35.649796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:35.649796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:35.649796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:35.649796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:35.650720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:35.650720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:35.650720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:35.650720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:35.650720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:35.650720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:35.650720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:35.650720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:35.650720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 304/607] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:35.651483 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553c9d1-1dc1-4293-89a1-39604bc37bee None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:35.651597 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553c9d1-1dc1-4293-89a1-39604bc37bee None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:35.651870 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553c9d1-1dc1-4293-89a1-39604bc37bee None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:35.653100 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 305/608] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:35.657716 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553c9d1-1dc1-4293-89a1-39604bc37bee None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:35.720755 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34feab47-5b03-4e2b-a2ea-de7bd9337fe9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:35.766664 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7264af-81f3-479c-a636-825540a21fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:35.767538 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7264af-81f3-479c-a636-825540a21fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:35.768216 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7264af-81f3-479c-a636-825540a21fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:35.768490 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7264af-81f3-479c-a636-825540a21fd7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:35.775426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553c9d1-1dc1-4293-89a1-39604bc37bee None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:35.782034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7264af-81f3-479c-a636-825540a21fd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:35.789109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-eb7264af-81f3-479c-a636-825540a21fd7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:35.789109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:35.789109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:35.789109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:35.789109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:35.789109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:35.789109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:35.789109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:35.789109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:35.789109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:35.789109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:35.789109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:35.789109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:35.789109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:35.789109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:35.789109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:35.790040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:35.790040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:35.790040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:35.790040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:35.790040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:35.790040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:35.790040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:35.790040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:35.790040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:35.790040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:35.790040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:35.790040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:35.790040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:35.790040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:35.790040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:35.790040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:35.790919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:35.790919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:35.790919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:35.790919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:35.790919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:35.790919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:35.790919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:35.790919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:35.790919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:35.790919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:35.790919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:35.790919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:35.791540 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 304/609] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:35.796536 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d6e80-3968-479b-b760-6a63be29bcce None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:35.796536 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d6e80-3968-479b-b760-6a63be29bcce None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:35.796536 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d6e80-3968-479b-b760-6a63be29bcce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:35.804057 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:35.821646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29571715-32bc-475b-8e2d-a67cc35daf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:35.822083 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-850d6e80-3968-479b-b760-6a63be29bcce None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:35.822284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29571715-32bc-475b-8e2d-a67cc35daf45 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:35.822625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29571715-32bc-475b-8e2d-a67cc35daf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:35.822942 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29571715-32bc-475b-8e2d-a67cc35daf45 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:35.823640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29571715-32bc-475b-8e2d-a67cc35daf45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:35.823949 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29571715-32bc-475b-8e2d-a67cc35daf45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:35.824211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-850d6e80-3968-479b-b760-6a63be29bcce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:35.825539 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29571715-32bc-475b-8e2d-a67cc35daf45 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:35.844349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 306/610] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:35.845305 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29571715-32bc-475b-8e2d-a67cc35daf45 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:35.845600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:35.918279 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 305/611] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:35.918279 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d6e80-3968-479b-b760-6a63be29bcce None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:35.982289 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29571715-32bc-475b-8e2d-a67cc35daf45 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:36.027761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844c6e3-48db-4839-b5bc-31eb5fe7c33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:36.028372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844c6e3-48db-4839-b5bc-31eb5fe7c33b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:36.028648 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844c6e3-48db-4839-b5bc-31eb5fe7c33b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:36.028917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844c6e3-48db-4839-b5bc-31eb5fe7c33b None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:36.029856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844c6e3-48db-4839-b5bc-31eb5fe7c33b None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:36.030160 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844c6e3-48db-4839-b5bc-31eb5fe7c33b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:36.031830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844c6e3-48db-4839-b5bc-31eb5fe7c33b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:36.041143 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d6e80-3968-479b-b760-6a63be29bcce None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:36.047142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 307/612] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:13:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:36.047142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844c6e3-48db-4839-b5bc-31eb5fe7c33b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:36.091279 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacf04c-b6b1-48ad-9c2b-e5001a5ad509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:36.091929 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacf04c-b6b1-48ad-9c2b-e5001a5ad509 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:36.092206 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacf04c-b6b1-48ad-9c2b-e5001a5ad509 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:36.092477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacf04c-b6b1-48ad-9c2b-e5001a5ad509 None admin] PATH_INFO: `/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/78506fea1c024b169a4b7ee86116658c/roles/838b081e674f4c0ab4a33bcf8e614dc5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:36.096813 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:36.111652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacf04c-b6b1-48ad-9c2b-e5001a5ad509 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d474b61f1fe4f17aacb7a3665315c79, user_id=78506fea1c024b169a4b7ee86116658c, role_id=838b081e674f4c0ab4a33bcf8e614dc5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:36.113378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacf04c-b6b1-48ad-9c2b-e5001a5ad509 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:36.136319 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 306/613] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:13:35 2022] PUT /identity/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/78506fea1c024b169a4b7ee86116658c/roles/838b081e674f4c0ab4a33bcf8e614dc5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:36.136319 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacf04c-b6b1-48ad-9c2b-e5001a5ad509 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:36.141747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:36.209190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844c6e3-48db-4839-b5bc-31eb5fe7c33b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:36.278372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972cbc82-bedb-4e95-9b9d-800954293ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:36.279136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cbc82-bedb-4e95-9b9d-800954293ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:36.279417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cbc82-bedb-4e95-9b9d-800954293ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:36.279677 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cbc82-bedb-4e95-9b9d-800954293ec9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:36.303632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972cbc82-bedb-4e95-9b9d-800954293ec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:36.312806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-972cbc82-bedb-4e95-9b9d-800954293ec9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:36.312806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:36.312806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:36.312806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:36.312806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:36.312806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:36.312806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:36.312806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:36.312806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:36.312806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:36.312806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:36.312806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:36.312806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:36.312806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:36.312806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:36.312806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:36.313787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:36.313787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:36.313787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:36.313787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:36.313787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:36.313787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:36.313787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:36.313787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:36.313787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:36.313787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:36.313787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:36.313787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:36.313787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:36.313787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:36.313787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:36.313787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:36.314618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:36.314618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:36.314618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:36.314618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:36.314618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:36.314618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:36.314618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:36.314618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:36.314618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:36.314618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:36.314618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:36.314618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:36.320067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 308/614] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:36.327348 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972cbc82-bedb-4e95-9b9d-800954293ec9 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:36.344146 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacf04c-b6b1-48ad-9c2b-e5001a5ad509 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:36.398333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ecf97-db95-43d1-8b13-0a4291f2cbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:36.399291 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ecf97-db95-43d1-8b13-0a4291f2cbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:36.399734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ecf97-db95-43d1-8b13-0a4291f2cbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:36.400162 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ecf97-db95-43d1-8b13-0a4291f2cbc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:36.411439 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ecf97-db95-43d1-8b13-0a4291f2cbc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:36.413131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ecf97-db95-43d1-8b13-0a4291f2cbc9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:36.429296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-386ecf97-db95-43d1-8b13-0a4291f2cbc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:36.429296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:36.429296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:36.429296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:36.429296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:36.429296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:36.429296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:36.429296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:36.429296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:36.429296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:36.429296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:36.429296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:36.429296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:36.429296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:36.429296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:36.429296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:36.430474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:36.430474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:36.430474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:36.430474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:36.430474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:36.430474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:36.430474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:36.430474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:36.430474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:36.430474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:36.430474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:36.430474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:36.430474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:36.430474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:36.430474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:36.430474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:36.430474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:36.431513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:36.431513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:36.431513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:36.433230 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 307/615] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:36.440055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ecf97-db95-43d1-8b13-0a4291f2cbc9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:36.455569 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972cbc82-bedb-4e95-9b9d-800954293ec9 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:36.457343 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:36.494109 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef362c-0eb0-4f38-af0c-ddead089e989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:36.495037 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef362c-0eb0-4f38-af0c-ddead089e989 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:36.495532 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef362c-0eb0-4f38-af0c-ddead089e989 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:36.495907 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef362c-0eb0-4f38-af0c-ddead089e989 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:36.496672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef362c-0eb0-4f38-af0c-ddead089e989 None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:36.497077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef362c-0eb0-4f38-af0c-ddead089e989 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:36.498740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef362c-0eb0-4f38-af0c-ddead089e989 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:36.500881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:36.509397 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 309/616] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:13:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:36.527693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef362c-0eb0-4f38-af0c-ddead089e989 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:36.558672 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ecf97-db95-43d1-8b13-0a4291f2cbc9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:36.593112 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd524047-9c96-4f75-bc0f-b443f68d14b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:36.593717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd524047-9c96-4f75-bc0f-b443f68d14b9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:36.593991 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd524047-9c96-4f75-bc0f-b443f68d14b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:36.594278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd524047-9c96-4f75-bc0f-b443f68d14b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:36.594924 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd524047-9c96-4f75-bc0f-b443f68d14b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:36.595222 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd524047-9c96-4f75-bc0f-b443f68d14b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:36.596620 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd524047-9c96-4f75-bc0f-b443f68d14b9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:36.624819 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 308/617] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:36.643300 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd524047-9c96-4f75-bc0f-b443f68d14b9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:36.681870 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef362c-0eb0-4f38-af0c-ddead089e989 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:36.719286 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b0a17-ff7c-4828-b4be-dda6d0c49f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:36.719915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b0a17-ff7c-4828-b4be-dda6d0c49f8a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:36.720254 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b0a17-ff7c-4828-b4be-dda6d0c49f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:36.720527 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b0a17-ff7c-4828-b4be-dda6d0c49f8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:36.732687 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b0a17-ff7c-4828-b4be-dda6d0c49f8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:36.734126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b0a17-ff7c-4828-b4be-dda6d0c49f8a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:36.750669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-9f0b0a17-ff7c-4828-b4be-dda6d0c49f8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:36.750669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:36.750669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:36.750669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:36.750669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:36.750669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:36.750669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:36.750669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:36.750669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:36.750669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:36.750669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:36.750669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:36.750669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:36.750669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:36.750669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:36.750669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:36.751654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:36.751654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:36.751654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:36.751654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:36.751654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:36.751654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:36.751654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:36.751654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:36.751654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:36.751654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:36.751654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:36.751654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:36.751654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:36.751654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:36.751654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:36.751654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:36.751654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:36.753067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:36.753067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:36.753067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:36.756556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 310/618] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:36.757845 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b0a17-ff7c-4828-b4be-dda6d0c49f8a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:36.778667 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:36.873846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b0a17-ff7c-4828-b4be-dda6d0c49f8a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:36.911326 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176ce72-173d-4ed7-928e-55755b5f373d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:36.912568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176ce72-173d-4ed7-928e-55755b5f373d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:36.912988 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176ce72-173d-4ed7-928e-55755b5f373d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:36.913398 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176ce72-173d-4ed7-928e-55755b5f373d None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:36.914174 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176ce72-173d-4ed7-928e-55755b5f373d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:36.914634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176ce72-173d-4ed7-928e-55755b5f373d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:36.916297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176ce72-173d-4ed7-928e-55755b5f373d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:36.924212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd524047-9c96-4f75-bc0f-b443f68d14b9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:36.933441 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 311/619] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:36.935132 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176ce72-173d-4ed7-928e-55755b5f373d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:36.949183 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:36.994236 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e820be0-3998-4cf1-9b22-0ec2ff5bad83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:36.995157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e820be0-3998-4cf1-9b22-0ec2ff5bad83 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:36.995578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e820be0-3998-4cf1-9b22-0ec2ff5bad83 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:36.995985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e820be0-3998-4cf1-9b22-0ec2ff5bad83 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:37.009722 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e820be0-3998-4cf1-9b22-0ec2ff5bad83 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:37.012945 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e820be0-3998-4cf1-9b22-0ec2ff5bad83 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:37.037256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7e820be0-3998-4cf1-9b22-0ec2ff5bad83 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:13:37.037256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:37.037256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:37.037256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:37.037256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:37.037256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:37.037256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:37.037256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:37.037256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:37.037256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:37.037256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:37.037256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:37.037256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:37.037256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:37.037256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:37.037256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:37.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:37.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:37.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:37.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:37.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:37.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:37.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:37.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:37.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:37.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:37.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:37.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:37.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:37.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:37.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:37.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:37.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:37.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:37.039746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:37.039746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:37.039746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:37.039746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:37.039746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:37.039746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:37.039746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:37.039746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:37.039746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:13:37.039746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:37.045658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 309/620] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:37.051684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e820be0-3998-4cf1-9b22-0ec2ff5bad83 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:37.058582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:37.101711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176ce72-173d-4ed7-928e-55755b5f373d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:37.138346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fd468-1f14-484b-b442-c0e1a60ffa9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:37.138914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fd468-1f14-484b-b442-c0e1a60ffa9b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:37.139419 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fd468-1f14-484b-b442-c0e1a60ffa9b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:37.139758 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fd468-1f14-484b-b442-c0e1a60ffa9b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:37.148063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fd468-1f14-484b-b442-c0e1a60ffa9b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:37.149687 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fd468-1f14-484b-b442-c0e1a60ffa9b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:37.162426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-ac2fd468-1f14-484b-b442-c0e1a60ffa9b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:13:37.162426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:37.162426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:37.162426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:37.162426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:37.162426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:37.162426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:37.162426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:37.162426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:37.162426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:37.162426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:37.162426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:37.162426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:37.162426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:37.162426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:37.162426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:37.163593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:37.163593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:37.163593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:37.163593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:37.163593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:37.163593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:37.163593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:37.163593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:37.163593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:37.163593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:37.163593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:37.163593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:37.163593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:37.163593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:37.163593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:37.163593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:37.163593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:37.163593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:37.164727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:37.164727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:37.164727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:37.164727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:37.164727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:37.164727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:37.164727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:37.164727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:37.164727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:13:37.164727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:37.170392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 312/621] 10.222.0.39 () {56 vars in 1093 bytes} [Mon Aug 8 02:13:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:37.174171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fd468-1f14-484b-b442-c0e1a60ffa9b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:37.261411 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e820be0-3998-4cf1-9b22-0ec2ff5bad83 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:37.278575 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fd468-1f14-484b-b442-c0e1a60ffa9b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:37.318689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee5ad5-7b8f-4324-b536-58f49e23d9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:37.319297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee5ad5-7b8f-4324-b536-58f49e23d9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:37.319577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee5ad5-7b8f-4324-b536-58f49e23d9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:37.319850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee5ad5-7b8f-4324-b536-58f49e23d9d7 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:37.320580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee5ad5-7b8f-4324-b536-58f49e23d9d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:37.321784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee5ad5-7b8f-4324-b536-58f49e23d9d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:37.323949 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee5ad5-7b8f-4324-b536-58f49e23d9d7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:37.330625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:37.341082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc39d9a-2974-47e8-9688-3b435db81e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:37.341691 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc39d9a-2974-47e8-9688-3b435db81e5a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:37.341982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc39d9a-2974-47e8-9688-3b435db81e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:37.342232 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc39d9a-2974-47e8-9688-3b435db81e5a None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:37.342986 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc39d9a-2974-47e8-9688-3b435db81e5a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:37.343288 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc39d9a-2974-47e8-9688-3b435db81e5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:37.344698 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc39d9a-2974-47e8-9688-3b435db81e5a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:37.362969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 313/622] 10.222.0.39 () {56 vars in 1133 bytes} [Mon Aug 8 02:13:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:37.368493 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee5ad5-7b8f-4324-b536-58f49e23d9d7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:37.378405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 310/623] 10.222.0.39 () {56 vars in 1135 bytes} [Mon Aug 8 02:13:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:37.386413 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc39d9a-2974-47e8-9688-3b435db81e5a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:37.454158 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:37.539121 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee5ad5-7b8f-4324-b536-58f49e23d9d7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:37.563643 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc39d9a-2974-47e8-9688-3b435db81e5a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:37.579822 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a7ae5-21ea-4ad1-854b-0c431c49c1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:37.580446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a7ae5-21ea-4ad1-854b-0c431c49c1b9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:37.580717 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a7ae5-21ea-4ad1-854b-0c431c49c1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:37.580959 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a7ae5-21ea-4ad1-854b-0c431c49c1b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:37.610465 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea301f35-9ce1-4e8d-845e-8e132f634fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:37.611999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea301f35-9ce1-4e8d-845e-8e132f634fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:37.612454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea301f35-9ce1-4e8d-845e-8e132f634fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:37.613126 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea301f35-9ce1-4e8d-845e-8e132f634fc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:37.626797 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea301f35-9ce1-4e8d-845e-8e132f634fc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:37.628270 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea301f35-9ce1-4e8d-845e-8e132f634fc6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:37.650262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:37.652996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ea301f35-9ce1-4e8d-845e-8e132f634fc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:37.652996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:37.652996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:37.652996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:37.652996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:37.652996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:37.652996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:37.652996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:37.652996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:37.652996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:37.652996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:37.652996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:37.652996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:37.652996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:37.652996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:37.652996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:37.654870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:37.654870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:37.654870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:37.654870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:37.654870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:37.654870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:37.654870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:37.654870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:37.654870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:37.654870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:37.654870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:37.654870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:37.654870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:37.654870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:37.654870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:37.654870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:37.654870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:37.656423 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:37.656423 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:37.656423 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:37.657686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 311/624] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:37.659442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1545426-73ec-404d-a94b-37641834e3ba None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:37.659582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a7ae5-21ea-4ad1-854b-0c431c49c1b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:37.659990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1545426-73ec-404d-a94b-37641834e3ba None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:37.660408 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1545426-73ec-404d-a94b-37641834e3ba None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:37.661239 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a7ae5-21ea-4ad1-854b-0c431c49c1b9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:37.666036 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 312/625] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:37.668157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc72e8-b80f-4051-a93f-f430125cc5f4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:37.668443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc72e8-b80f-4051-a93f-f430125cc5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:37.668719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc72e8-b80f-4051-a93f-f430125cc5f4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:37.670025 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 313/626] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:37.671053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc72e8-b80f-4051-a93f-f430125cc5f4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:37.761451 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:37.771830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 314/627] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:37.776491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6274e-14a3-4747-8227-e5479e802ddc None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:37.776712 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6274e-14a3-4747-8227-e5479e802ddc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:37.776960 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6274e-14a3-4747-8227-e5479e802ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:37.779102 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc72e8-b80f-4051-a93f-f430125cc5f4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:37.794689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-e9b6274e-14a3-4747-8227-e5479e802ddc None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:37.795881 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e9b6274e-14a3-4747-8227-e5479e802ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:37.816841 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3eebf-3dbb-4cb2-b7df-0bed866764d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:37.817984 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3eebf-3dbb-4cb2-b7df-0bed866764d7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:37.818748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3eebf-3dbb-4cb2-b7df-0bed866764d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:37.819027 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3eebf-3dbb-4cb2-b7df-0bed866764d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:37.819757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3eebf-3dbb-4cb2-b7df-0bed866764d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:37.819978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3eebf-3dbb-4cb2-b7df-0bed866764d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:37.821358 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3eebf-3dbb-4cb2-b7df-0bed866764d7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:37.833626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 314/628] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:37.835527 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256318da-8bbf-428e-b5a7-fc144026d5df None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:37.835812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256318da-8bbf-428e-b5a7-fc144026d5df None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:37.836085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256318da-8bbf-428e-b5a7-fc144026d5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:37.852948 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-256318da-8bbf-428e-b5a7-fc144026d5df None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:37.854340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-256318da-8bbf-428e-b5a7-fc144026d5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:37.874669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 315/629] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:37.875593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6274e-14a3-4747-8227-e5479e802ddc None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:37.905945 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:37.946350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 315/630] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:37.946350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256318da-8bbf-428e-b5a7-fc144026d5df None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:37.984387 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6274e-14a3-4747-8227-e5479e802ddc None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:38.020866 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdece4d5-6512-43d5-b030-7a1ce6405f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:38.021751 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdece4d5-6512-43d5-b030-7a1ce6405f0c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:38.022166 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdece4d5-6512-43d5-b030-7a1ce6405f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:38.022599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdece4d5-6512-43d5-b030-7a1ce6405f0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:38.023688 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:38.033780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdece4d5-6512-43d5-b030-7a1ce6405f0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:38.036243 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdece4d5-6512-43d5-b030-7a1ce6405f0c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:38.050668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-cdece4d5-6512-43d5-b030-7a1ce6405f0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:38.050668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:38.050668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:38.050668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:38.050668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:38.050668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:38.050668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:38.050668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:38.050668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:38.050668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:38.050668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:38.050668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:38.050668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:38.050668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:38.050668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:38.050668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:38.052165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:38.052165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:38.052165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:38.052165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:38.052165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:38.052165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:38.052165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:38.052165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:38.052165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:38.052165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:38.052165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:38.052165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:38.052165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:38.052165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:38.052165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:38.052165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:38.052165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:38.053698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:38.053698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:38.053698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:38.053905 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256318da-8bbf-428e-b5a7-fc144026d5df None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:38.053980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 316/631] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:38.053980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdece4d5-6512-43d5-b030-7a1ce6405f0c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:38.095008 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838a5b9-bd5c-4318-99fb-23c833f74835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:38.095634 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838a5b9-bd5c-4318-99fb-23c833f74835 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:38.095913 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838a5b9-bd5c-4318-99fb-23c833f74835 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:38.096182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838a5b9-bd5c-4318-99fb-23c833f74835 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:38.103861 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838a5b9-bd5c-4318-99fb-23c833f74835 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:38.105307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838a5b9-bd5c-4318-99fb-23c833f74835 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:38.117852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3838a5b9-bd5c-4318-99fb-23c833f74835 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:38.117852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:38.117852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:38.117852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:38.117852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:38.117852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:38.117852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:38.117852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:38.117852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:38.117852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:38.117852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:38.117852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:38.117852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:38.117852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:38.117852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:38.118964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:38.118964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:38.118964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:38.118964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:38.118964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:38.118964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:38.118964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:38.118964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:38.118964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:38.118964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:38.118964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:38.118964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:38.118964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:38.118964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:38.118964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:38.118964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:38.118964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:38.119838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:38.119838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:38.119838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:38.119838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:38.119838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:38.119838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:38.119838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:38.119838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:38.123710 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 316/632] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:38.124550 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838a5b9-bd5c-4318-99fb-23c833f74835 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:38.133594 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:38.157866 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdece4d5-6512-43d5-b030-7a1ce6405f0c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:38.193620 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b583d-99e5-46dc-a9e5-0cef5aabf43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:38.194237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b583d-99e5-46dc-a9e5-0cef5aabf43e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:38.194548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b583d-99e5-46dc-a9e5-0cef5aabf43e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:38.194816 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b583d-99e5-46dc-a9e5-0cef5aabf43e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:38.205527 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b583d-99e5-46dc-a9e5-0cef5aabf43e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:38.210665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-b37b583d-99e5-46dc-a9e5-0cef5aabf43e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:38.210665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:38.210665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:38.210665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:38.210665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:38.210665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:38.210665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:38.210665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:38.210665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:38.210665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:38.210665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:38.210665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:38.210665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:38.210665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:38.210665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:38.210665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:38.211825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:38.211825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:38.211825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:38.211825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:38.211825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:38.211825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:38.211825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:38.211825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:38.211825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:38.211825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:38.211825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:38.211825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:38.211825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:38.211825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:38.211825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:38.211825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:38.212646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:38.212646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:38.212646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:38.212646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:38.212646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:38.212646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:38.212646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:38.212646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:38.212646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:38.212646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:38.212646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:38.212646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:38.212646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 317/633] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:38.212646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b583d-99e5-46dc-a9e5-0cef5aabf43e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:38.232945 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838a5b9-bd5c-4318-99fb-23c833f74835 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:38.267934 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357924dd-aed5-4d92-ac24-22eaca8dd8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:38.268652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357924dd-aed5-4d92-ac24-22eaca8dd8ae None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:38.268996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357924dd-aed5-4d92-ac24-22eaca8dd8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:38.269330 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357924dd-aed5-4d92-ac24-22eaca8dd8ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:38.271466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:38.281639 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357924dd-aed5-4d92-ac24-22eaca8dd8ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:38.287295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-357924dd-aed5-4d92-ac24-22eaca8dd8ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:38.287295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:38.287295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:38.287295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:38.287295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:38.287295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:38.287295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:38.287295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:38.287295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:38.287295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:38.287295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:38.287295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:38.287295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:38.287295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:38.287295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:38.287295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:38.288431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:38.288431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:38.288431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:38.288431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:38.288431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:38.288431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:38.288431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:38.288431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:38.288431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:38.288431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:38.288431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:38.288431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:38.288431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:38.288431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:38.288431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:38.288431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:38.289464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:38.289464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:38.289464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:38.289464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:38.289464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:38.289464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:38.289464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:38.289464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:38.289464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:38.289464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:38.289464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:38.289464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:38.289464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 317/634] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:38.289464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357924dd-aed5-4d92-ac24-22eaca8dd8ae None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:38.324600 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b583d-99e5-46dc-a9e5-0cef5aabf43e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:38.368809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82526f-acb8-4d3c-97c1-9d986f102a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:38.369517 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82526f-acb8-4d3c-97c1-9d986f102a14 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:38.369861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82526f-acb8-4d3c-97c1-9d986f102a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:38.370195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82526f-acb8-4d3c-97c1-9d986f102a14 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:38.371449 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82526f-acb8-4d3c-97c1-9d986f102a14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:38.371837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82526f-acb8-4d3c-97c1-9d986f102a14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:38.373763 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82526f-acb8-4d3c-97c1-9d986f102a14 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:38.377425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:38.388240 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 318/635] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:38.389077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82526f-acb8-4d3c-97c1-9d986f102a14 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:38.401815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357924dd-aed5-4d92-ac24-22eaca8dd8ae None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:38.445692 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad280ee2-b977-4773-8e03-5c2b41b96bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:38.446335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad280ee2-b977-4773-8e03-5c2b41b96bfc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:38.446618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad280ee2-b977-4773-8e03-5c2b41b96bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:38.446888 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad280ee2-b977-4773-8e03-5c2b41b96bfc None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:38.447574 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad280ee2-b977-4773-8e03-5c2b41b96bfc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:38.447879 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad280ee2-b977-4773-8e03-5c2b41b96bfc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:38.449334 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad280ee2-b977-4773-8e03-5c2b41b96bfc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:38.463941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 318/636] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:38.464753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad280ee2-b977-4773-8e03-5c2b41b96bfc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:38.495204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82526f-acb8-4d3c-97c1-9d986f102a14 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:38.521352 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:38.531410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a0fef-9116-4457-99ef-534d79b3e2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:38.531949 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a0fef-9116-4457-99ef-534d79b3e2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:38.532230 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a0fef-9116-4457-99ef-534d79b3e2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:38.532508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a0fef-9116-4457-99ef-534d79b3e2d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:38.533167 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a0fef-9116-4457-99ef-534d79b3e2d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:38.533470 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a0fef-9116-4457-99ef-534d79b3e2d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:38.534928 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a0fef-9116-4457-99ef-534d79b3e2d5 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:38.543305 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 319/637] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:13:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:38.544170 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a0fef-9116-4457-99ef-534d79b3e2d5 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:38.570959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad280ee2-b977-4773-8e03-5c2b41b96bfc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:38.600803 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:38.605073 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648fc98-e2d9-47d4-9d28-43c6996b0e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:38.605662 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648fc98-e2d9-47d4-9d28-43c6996b0e9c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:38.605932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648fc98-e2d9-47d4-9d28-43c6996b0e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:38.606195 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648fc98-e2d9-47d4-9d28-43c6996b0e9c None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:38.606870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648fc98-e2d9-47d4-9d28-43c6996b0e9c None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:38.607166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648fc98-e2d9-47d4-9d28-43c6996b0e9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:38.608515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648fc98-e2d9-47d4-9d28-43c6996b0e9c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:38.616214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 319/638] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:13:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:38.617409 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9542dda-a761-4d42-913b-7e6f361b17da None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:38.617693 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9542dda-a761-4d42-913b-7e6f361b17da None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:38.617960 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9542dda-a761-4d42-913b-7e6f361b17da None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:38.619165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 320/639] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:38.619822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9542dda-a761-4d42-913b-7e6f361b17da None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:38.645950 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a0fef-9116-4457-99ef-534d79b3e2d5 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:38.683944 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60723d-9748-47c6-a139-463bb3132d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:38.685163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60723d-9748-47c6-a139-463bb3132d1a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:38.685627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60723d-9748-47c6-a139-463bb3132d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:38.686133 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60723d-9748-47c6-a139-463bb3132d1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:38.693275 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60723d-9748-47c6-a139-463bb3132d1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:38.694745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60723d-9748-47c6-a139-463bb3132d1a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:38.705476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-5d60723d-9748-47c6-a139-463bb3132d1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:38.705476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:38.705476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:38.705476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:38.705476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:38.705476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:38.705476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:38.705476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:38.705476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:38.705476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:38.705476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:38.705476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:38.705476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:38.705476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:38.705476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:38.706655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:38.706655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:38.706655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:38.706655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:38.706655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:38.706655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:38.706655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:38.706655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:38.706655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:38.706655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:38.706655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:38.706655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:38.706655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:38.706655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:38.706655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:38.706655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:38.706655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:38.708689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:38.708689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:38.708689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:38.708689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:38.708689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:38.708689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:38.708689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:38.708689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:38.708689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 320/640] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:38.708689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60723d-9748-47c6-a139-463bb3132d1a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:38.725040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9542dda-a761-4d42-913b-7e6f361b17da None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:38.741895 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:38.759478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00704c1a-f81a-4c06-b9e8-f6f6d7a23dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:38.760126 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00704c1a-f81a-4c06-b9e8-f6f6d7a23dce None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:38.760403 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00704c1a-f81a-4c06-b9e8-f6f6d7a23dce None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:38.760674 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00704c1a-f81a-4c06-b9e8-f6f6d7a23dce None admin] PATH_INFO: `/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/ac60b4d621234225830c94edfac400e2/roles/838b081e674f4c0ab4a33bcf8e614dc5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:38.775043 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00704c1a-f81a-4c06-b9e8-f6f6d7a23dce None admin] RBAC: Authorizing `identity:create_grant(project_id=7d474b61f1fe4f17aacb7a3665315c79, user_id=ac60b4d621234225830c94edfac400e2, role_id=838b081e674f4c0ab4a33bcf8e614dc5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:38.776475 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00704c1a-f81a-4c06-b9e8-f6f6d7a23dce None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:38.797395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 321/641] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:13:38 2022] PUT /identity/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/ac60b4d621234225830c94edfac400e2/roles/838b081e674f4c0ab4a33bcf8e614dc5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:38.797395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00704c1a-f81a-4c06-b9e8-f6f6d7a23dce None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:38.816076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60723d-9748-47c6-a139-463bb3132d1a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:38.836039 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:38.854939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec898bc-e575-4bf4-bd78-abd35ac0afce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:38.855654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec898bc-e575-4bf4-bd78-abd35ac0afce None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:38.855935 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec898bc-e575-4bf4-bd78-abd35ac0afce None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:38.856215 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec898bc-e575-4bf4-bd78-abd35ac0afce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:38.868121 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec898bc-e575-4bf4-bd78-abd35ac0afce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:38.869613 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec898bc-e575-4bf4-bd78-abd35ac0afce None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:38.887068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-6ec898bc-e575-4bf4-bd78-abd35ac0afce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:38.887068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:38.887068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:38.887068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:38.887068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:38.887068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:38.887068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:38.887068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:38.887068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:38.887068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:38.887068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:38.887068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:38.887068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:38.887068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:38.887068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:38.887068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:38.888141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:38.888141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:38.888141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:38.888141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:38.888141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:38.888141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:38.888141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:38.888141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:38.888141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:38.888141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:38.888141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:38.888141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:38.888141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:38.888141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:38.888141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:38.888141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:38.888141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:38.894580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:38.894580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:38.894580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:38.894580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 321/642] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:38.894580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c38b6d-ca70-4af5-b154-b539a8a2cc0d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:38.894580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c38b6d-ca70-4af5-b154-b539a8a2cc0d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:38.894580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c38b6d-ca70-4af5-b154-b539a8a2cc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:38.912271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-90c38b6d-ca70-4af5-b154-b539a8a2cc0d None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:38.912476 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00704c1a-f81a-4c06-b9e8-f6f6d7a23dce None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:38.913493 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-90c38b6d-ca70-4af5-b154-b539a8a2cc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:38.955207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff81333-1b91-4622-9d74-9e5ad4bca442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:38.955835 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff81333-1b91-4622-9d74-9e5ad4bca442 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:38.956109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff81333-1b91-4622-9d74-9e5ad4bca442 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:38.956376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff81333-1b91-4622-9d74-9e5ad4bca442 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:38.967084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff81333-1b91-4622-9d74-9e5ad4bca442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:38.968536 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff81333-1b91-4622-9d74-9e5ad4bca442 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:38.988320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-cff81333-1b91-4622-9d74-9e5ad4bca442 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:38.988320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:38.988320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:38.988320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:38.988320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:38.988320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:38.988320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:38.988320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:38.988320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:38.988320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:38.988320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:38.988320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:38.988320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:38.988320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:38.988320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:38.988320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:38.989366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:38.989366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:38.989366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:38.989366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:38.989366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:38.989366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:38.989366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:38.989366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:38.989366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:38.989366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:38.989366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:38.989366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:38.989366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:38.989366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:38.989366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:38.989366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:38.989366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:38.990864 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:38.990864 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:38.990864 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:38.990864 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 322/643] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:38.991093 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 322/644] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:38.991166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff81333-1b91-4622-9d74-9e5ad4bca442 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:38.991232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c38b6d-ca70-4af5-b154-b539a8a2cc0d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:39.013463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:39.099835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:39.103351 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c38b6d-ca70-4af5-b154-b539a8a2cc0d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:39.108567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff81333-1b91-4622-9d74-9e5ad4bca442 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:39.138174 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c34042-6ec4-48d7-bd78-36d87261e713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:39.138794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c34042-6ec4-48d7-bd78-36d87261e713 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:39.139071 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c34042-6ec4-48d7-bd78-36d87261e713 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:39.139338 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c34042-6ec4-48d7-bd78-36d87261e713 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:39.139974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c34042-6ec4-48d7-bd78-36d87261e713 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:39.140276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c34042-6ec4-48d7-bd78-36d87261e713 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:39.141739 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c34042-6ec4-48d7-bd78-36d87261e713 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:39.144734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0ad9d-aed9-4bd0-a731-86e9afff71a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:39.145881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0ad9d-aed9-4bd0-a731-86e9afff71a0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:39.146162 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0ad9d-aed9-4bd0-a731-86e9afff71a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:39.146462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0ad9d-aed9-4bd0-a731-86e9afff71a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:39.147151 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0ad9d-aed9-4bd0-a731-86e9afff71a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:39.147459 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0ad9d-aed9-4bd0-a731-86e9afff71a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:39.149030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0ad9d-aed9-4bd0-a731-86e9afff71a0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:39.158469 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 323/645] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:39.158469 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c34042-6ec4-48d7-bd78-36d87261e713 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:39.168760 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 323/646] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:39.168760 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0ad9d-aed9-4bd0-a731-86e9afff71a0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:39.253934 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:39.269965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c34042-6ec4-48d7-bd78-36d87261e713 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:39.278779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0ad9d-aed9-4bd0-a731-86e9afff71a0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:39.305199 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e88073-33a1-4c3b-a356-25d2b11ddbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:39.305805 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e88073-33a1-4c3b-a356-25d2b11ddbdd None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:39.306112 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e88073-33a1-4c3b-a356-25d2b11ddbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:39.306389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e88073-33a1-4c3b-a356-25d2b11ddbdd None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:39.307033 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e88073-33a1-4c3b-a356-25d2b11ddbdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:39.307342 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e88073-33a1-4c3b-a356-25d2b11ddbdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:39.308708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e88073-33a1-4c3b-a356-25d2b11ddbdd None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:39.315065 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bacbb40-e85f-40cb-80d0-67371c4aff06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:39.315679 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bacbb40-e85f-40cb-80d0-67371c4aff06 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:39.316063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bacbb40-e85f-40cb-80d0-67371c4aff06 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:39.316329 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bacbb40-e85f-40cb-80d0-67371c4aff06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:39.321989 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 324/647] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:39.327346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e88073-33a1-4c3b-a356-25d2b11ddbdd None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:39.327485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bacbb40-e85f-40cb-80d0-67371c4aff06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:39.332001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-9bacbb40-e85f-40cb-80d0-67371c4aff06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:39.332001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:39.332001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:39.332001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:39.332001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:39.332001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:39.332001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:39.332001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:39.332001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:39.332001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:39.332001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:39.332001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:39.332001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:39.332001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:39.332001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:39.332001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:39.332936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:39.332936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:39.332936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:39.332936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:39.332936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:39.332936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:39.332936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:39.332936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:39.332936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:39.332936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:39.332936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:39.332936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:39.332936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:39.332936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:39.332936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:39.332936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:39.333766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:39.333766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:39.333766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:39.333766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:39.333766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:39.333766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:39.333766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:39.333766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:39.333766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:39.333766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:39.333766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:39.333766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:39.333766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 324/648] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:39.334484 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bacbb40-e85f-40cb-80d0-67371c4aff06 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:39.352540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:39.438894 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e88073-33a1-4c3b-a356-25d2b11ddbdd None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:39.445951 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bacbb40-e85f-40cb-80d0-67371c4aff06 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:39.475652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95575fa9-01de-441c-88cd-8a894275fd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:39.476267 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95575fa9-01de-441c-88cd-8a894275fd67 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:39.476540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95575fa9-01de-441c-88cd-8a894275fd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:39.476805 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95575fa9-01de-441c-88cd-8a894275fd67 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:39.481752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c6fa1-30f1-412a-9a3c-eb16ef782faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:39.482374 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c6fa1-30f1-412a-9a3c-eb16ef782faf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:39.482648 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c6fa1-30f1-412a-9a3c-eb16ef782faf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:39.482914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c6fa1-30f1-412a-9a3c-eb16ef782faf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:39.483633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c6fa1-30f1-412a-9a3c-eb16ef782faf None admin] RBAC: Adding query filter params (role.id=838b081e674f4c0ab4a33bcf8e614dc5, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=c4238566a9c248ffbfa314f47f02670c) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:39.483940 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c6fa1-30f1-412a-9a3c-eb16ef782faf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:39.484180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95575fa9-01de-441c-88cd-8a894275fd67 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:39.485382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c6fa1-30f1-412a-9a3c-eb16ef782faf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:39.485587 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95575fa9-01de-441c-88cd-8a894275fd67 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:39.502556 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 325/649] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:39 2022] GET /identity/v3/role_assignments?user.id=c4238566a9c248ffbfa314f47f02670c&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=838b081e674f4c0ab4a33bcf8e614dc5 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:39.502556 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c6fa1-30f1-412a-9a3c-eb16ef782faf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:39.502556 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:39.509496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-95575fa9-01de-441c-88cd-8a894275fd67 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 02:13:39.509496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:39.509496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:39.509496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:39.509496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:39.509496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:39.509496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:39.509496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:39.509496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:39.509496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:39.509496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:39.509496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:39.509496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:39.509496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:39.509496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:39.509496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:39.510647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:39.510647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:39.510647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:39.510647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:39.510647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:39.510647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:39.510647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:39.510647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:39.510647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:39.510647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:39.510647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:39.510647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:39.510647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:39.510647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:39.510647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:39.510647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:39.510647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:39.510647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:39.511567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:39.511567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:39.511567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:39.511567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:39.511567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:39.511567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:39.511567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:39.511567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:39.511567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 02:13:39.511567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:39.511567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 325/650] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:39.512168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95575fa9-01de-441c-88cd-8a894275fd67 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:39.624290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c6fa1-30f1-412a-9a3c-eb16ef782faf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:39.639521 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:39.663009 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239eaa7-6965-4bb8-b7c5-566d13c55db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:39.663633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239eaa7-6965-4bb8-b7c5-566d13c55db7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:39.663908 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239eaa7-6965-4bb8-b7c5-566d13c55db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:39.664175 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239eaa7-6965-4bb8-b7c5-566d13c55db7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:39.669833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95575fa9-01de-441c-88cd-8a894275fd67 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:39.672883 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239eaa7-6965-4bb8-b7c5-566d13c55db7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:39.675148 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239eaa7-6965-4bb8-b7c5-566d13c55db7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:39.690992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-8239eaa7-6965-4bb8-b7c5-566d13c55db7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:13:39.690992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:39.690992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:39.690992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:39.690992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:39.690992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:39.690992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:39.690992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:39.690992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:39.690992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:39.690992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:39.690992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:39.690992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:39.690992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:39.690992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:39.690992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:39.692026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:39.692026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:39.692026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:39.692026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:39.692026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:39.692026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:39.692026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:39.692026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:39.692026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:39.692026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:39.692026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:39.692026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:39.692026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:39.692026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:39.692026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:39.692026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:39.692026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:39.692026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:39.692970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:39.692970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:39.692970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:39.692970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:39.692970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:39.692970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:39.692970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:39.692970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:39.692970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:13:39.692970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:39.692970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 326/651] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:39.693571 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239eaa7-6965-4bb8-b7c5-566d13c55db7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:39.713115 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97764a-5af3-49fe-83fe-4e28b31f3eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:39.713724 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97764a-5af3-49fe-83fe-4e28b31f3eea None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:39.714005 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97764a-5af3-49fe-83fe-4e28b31f3eea None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:39.714357 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97764a-5af3-49fe-83fe-4e28b31f3eea None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:39.715032 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97764a-5af3-49fe-83fe-4e28b31f3eea None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:39.715339 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97764a-5af3-49fe-83fe-4e28b31f3eea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:39.716709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97764a-5af3-49fe-83fe-4e28b31f3eea None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:39.725728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 326/652] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:13:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:39.726574 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97764a-5af3-49fe-83fe-4e28b31f3eea None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:39.767827 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:39.813381 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239eaa7-6965-4bb8-b7c5-566d13c55db7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:39.847834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf980c-5192-4833-8f55-dd76526879d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:39.848959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf980c-5192-4833-8f55-dd76526879d4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:39.849229 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97764a-5af3-49fe-83fe-4e28b31f3eea None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:39.849363 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf980c-5192-4833-8f55-dd76526879d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:39.849527 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf980c-5192-4833-8f55-dd76526879d4 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:39.850199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf980c-5192-4833-8f55-dd76526879d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:39.850542 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf980c-5192-4833-8f55-dd76526879d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:39.851973 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf980c-5192-4833-8f55-dd76526879d4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:39.872264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 327/653] 10.222.0.39 () {56 vars in 1137 bytes} [Mon Aug 8 02:13:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:39.876787 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf980c-5192-4833-8f55-dd76526879d4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:39.896883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74559b-d641-4fbd-b4d0-b118cf04aa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:39.897897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74559b-d641-4fbd-b4d0-b118cf04aa7d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:39.898517 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74559b-d641-4fbd-b4d0-b118cf04aa7d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:39.899015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74559b-d641-4fbd-b4d0-b118cf04aa7d None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:39.900584 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74559b-d641-4fbd-b4d0-b118cf04aa7d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:39.901270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74559b-d641-4fbd-b4d0-b118cf04aa7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:39.903391 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74559b-d641-4fbd-b4d0-b118cf04aa7d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:39.924982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:39.932103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 327/654] 10.222.0.39 () {56 vars in 1137 bytes} [Mon Aug 8 02:13:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:39.933770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74559b-d641-4fbd-b4d0-b118cf04aa7d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:39.987545 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf980c-5192-4833-8f55-dd76526879d4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:40.010020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:40.026505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5aecfc-329a-4d2f-90f7-57c7545bedb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:40.027235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5aecfc-329a-4d2f-90f7-57c7545bedb8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:40.027575 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5aecfc-329a-4d2f-90f7-57c7545bedb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:40.027938 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5aecfc-329a-4d2f-90f7-57c7545bedb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:40.039432 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5aecfc-329a-4d2f-90f7-57c7545bedb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:40.041007 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5aecfc-329a-4d2f-90f7-57c7545bedb8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:40.059492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6d5aecfc-329a-4d2f-90f7-57c7545bedb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:40.059492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:40.059492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:40.059492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:40.059492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:40.059492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:40.059492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:40.059492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:40.059492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:40.059492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:40.059492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:40.059492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:40.059492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:40.059492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:40.059492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:40.059492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:40.060713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:40.060713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:40.060713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:40.060713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:40.060713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:40.060713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:40.060713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:40.060713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:40.060713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:40.060713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:40.060713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:40.060713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:40.060713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:40.060713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:40.060713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:40.060713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:40.060713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:40.061593 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:40.061593 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:40.061593 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:40.067495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 328/655] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:40.067495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5aecfc-329a-4d2f-90f7-57c7545bedb8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:40.083556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74559b-d641-4fbd-b4d0-b118cf04aa7d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:40.129197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3ca4b-65cb-472b-a838-d7bc60574463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:40.130358 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3ca4b-65cb-472b-a838-d7bc60574463 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:40.131169 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3ca4b-65cb-472b-a838-d7bc60574463 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:40.131933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3ca4b-65cb-472b-a838-d7bc60574463 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:40.180926 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5aecfc-329a-4d2f-90f7-57c7545bedb8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:40.207209 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:40.210432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3ca4b-65cb-472b-a838-d7bc60574463 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:40.212246 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3ca4b-65cb-472b-a838-d7bc60574463 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:40.218540 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cbe2a-d22a-40be-babe-ad0bf63294a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:40.219235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cbe2a-d22a-40be-babe-ad0bf63294a9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:40.219519 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cbe2a-d22a-40be-babe-ad0bf63294a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:40.219793 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cbe2a-d22a-40be-babe-ad0bf63294a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:40.266020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:40.297368 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cbe2a-d22a-40be-babe-ad0bf63294a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:40.299274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cbe2a-d22a-40be-babe-ad0bf63294a9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:40.314996 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 328/656] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:40.315923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3ca4b-65cb-472b-a838-d7bc60574463 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:40.416421 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 329/657] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:40.417631 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cbe2a-d22a-40be-babe-ad0bf63294a9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:40.422546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:40.425024 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3ca4b-65cb-472b-a838-d7bc60574463 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:40.467510 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5ee0c-0efe-4398-8c6d-7073bb0b62ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:40.468166 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5ee0c-0efe-4398-8c6d-7073bb0b62ad None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:40.468512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5ee0c-0efe-4398-8c6d-7073bb0b62ad None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:40.468850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5ee0c-0efe-4398-8c6d-7073bb0b62ad None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:40.469669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5ee0c-0efe-4398-8c6d-7073bb0b62ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:40.469923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5ee0c-0efe-4398-8c6d-7073bb0b62ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:40.471379 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5ee0c-0efe-4398-8c6d-7073bb0b62ad None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:40.483251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 329/658] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:40.484238 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5ee0c-0efe-4398-8c6d-7073bb0b62ad None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:40.499386 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:40.535394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cbe2a-d22a-40be-babe-ad0bf63294a9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:40.573273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f6604-754a-4767-97fe-942e05eb3a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:40.574123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f6604-754a-4767-97fe-942e05eb3a2a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:40.574657 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f6604-754a-4767-97fe-942e05eb3a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:40.575148 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f6604-754a-4767-97fe-942e05eb3a2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:40.587079 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f6604-754a-4767-97fe-942e05eb3a2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:40.587337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5ee0c-0efe-4398-8c6d-7073bb0b62ad None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:40.589108 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f6604-754a-4767-97fe-942e05eb3a2a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:40.605267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a61f6604-754a-4767-97fe-942e05eb3a2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:40.605267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:40.605267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:40.605267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:40.605267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:40.605267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:40.605267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:40.605267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:40.605267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:40.605267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:40.605267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:40.605267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:40.605267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:40.605267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:40.605267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:40.605267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:40.606277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:40.606277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:40.606277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:40.606277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:40.606277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:40.606277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:40.606277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:40.606277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:40.606277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:40.606277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:40.606277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:40.606277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:40.606277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:40.606277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:40.606277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:40.606277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:40.606277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:40.607176 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:40.607176 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:40.607176 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:40.609341 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 330/659] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:40.610478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f6604-754a-4767-97fe-942e05eb3a2a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:40.622221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594a8d5-d587-42ac-8d3c-a100d2ba31cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:40.622872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594a8d5-d587-42ac-8d3c-a100d2ba31cc None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:40.623144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594a8d5-d587-42ac-8d3c-a100d2ba31cc None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:40.623411 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594a8d5-d587-42ac-8d3c-a100d2ba31cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:40.634946 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594a8d5-d587-42ac-8d3c-a100d2ba31cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:40.636380 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594a8d5-d587-42ac-8d3c-a100d2ba31cc None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:40.651381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-4594a8d5-d587-42ac-8d3c-a100d2ba31cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:40.651381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:40.651381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:40.651381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:40.651381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:40.651381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:40.651381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:40.651381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:40.651381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:40.651381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:40.651381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:40.651381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:40.651381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:40.651381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:40.651381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:40.651381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:40.652784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:40.652784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:40.652784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:40.652784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:40.652784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:40.652784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:40.652784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:40.652784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:40.652784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:40.652784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:40.652784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:40.652784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:40.652784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:40.652784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:40.652784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:40.652784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:40.652784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:40.653705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:40.653705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:40.653705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:40.656958 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 330/660] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:40.657793 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594a8d5-d587-42ac-8d3c-a100d2ba31cc None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:40.673830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:40.729574 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f6604-754a-4767-97fe-942e05eb3a2a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:40.756476 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:40.762508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594a8d5-d587-42ac-8d3c-a100d2ba31cc None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:40.770555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0aa82f-f6fd-47b5-9251-fb8c0d657d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:40.771194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0aa82f-f6fd-47b5-9251-fb8c0d657d41 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:40.771473 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0aa82f-f6fd-47b5-9251-fb8c0d657d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:40.771744 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0aa82f-f6fd-47b5-9251-fb8c0d657d41 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:40.782992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0aa82f-f6fd-47b5-9251-fb8c0d657d41 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:40.784810 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0aa82f-f6fd-47b5-9251-fb8c0d657d41 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:40.796667 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22c6dc-f441-4da8-956b-24737ab756a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:40.797260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22c6dc-f441-4da8-956b-24737ab756a6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:40.797536 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22c6dc-f441-4da8-956b-24737ab756a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:40.797855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22c6dc-f441-4da8-956b-24737ab756a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:40.798656 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22c6dc-f441-4da8-956b-24737ab756a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:40.798953 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22c6dc-f441-4da8-956b-24737ab756a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:40.799512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-bd0aa82f-f6fd-47b5-9251-fb8c0d657d41 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 02:13:40.799512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:40.799512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:40.799512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:40.799512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:40.799512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:40.799512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:40.799512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:40.799512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:40.799512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:40.799512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:40.799512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:40.799512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:40.799512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:40.799512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:40.799512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:40.800502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:40.800502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:40.800502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:40.800502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:40.800502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:40.800502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:40.800502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:40.800502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:40.800502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:40.800502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:40.800502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:40.800502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:40.800502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:40.800502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:40.800502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:40.800502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:40.800502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:40.800502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:40.801969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:40.801969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:40.801969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:40.801969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:40.801969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:40.801969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:40.801969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:40.801969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:40.801969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 02:13:40.801969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:40.802547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22c6dc-f441-4da8-956b-24737ab756a6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:40.802627 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 331/661] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:40.802627 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0aa82f-f6fd-47b5-9251-fb8c0d657d41 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:40.814562 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 331/662] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:40.815464 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22c6dc-f441-4da8-956b-24737ab756a6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:40.893448 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:40.912273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0aa82f-f6fd-47b5-9251-fb8c0d657d41 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:40.927347 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22c6dc-f441-4da8-956b-24737ab756a6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:40.948964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3e00f-d497-4b6f-9663-7c4a5e96c3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:40.949975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3e00f-d497-4b6f-9663-7c4a5e96c3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:40.950272 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3e00f-d497-4b6f-9663-7c4a5e96c3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:40.950551 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3e00f-d497-4b6f-9663-7c4a5e96c3d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:40.951194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3e00f-d497-4b6f-9663-7c4a5e96c3d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:40.951506 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3e00f-d497-4b6f-9663-7c4a5e96c3d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:40.952869 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3e00f-d497-4b6f-9663-7c4a5e96c3d1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:40.964076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a40a81-9413-4684-a877-1d33839c4e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:40.964378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 332/663] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:40.965092 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a40a81-9413-4684-a877-1d33839c4e02 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:40.965203 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3e00f-d497-4b6f-9663-7c4a5e96c3d1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:40.965658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a40a81-9413-4684-a877-1d33839c4e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:40.965938 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a40a81-9413-4684-a877-1d33839c4e02 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:40.966704 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a40a81-9413-4684-a877-1d33839c4e02 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:40.967017 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a40a81-9413-4684-a877-1d33839c4e02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:40.968423 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a40a81-9413-4684-a877-1d33839c4e02 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:40.987125 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 332/664] 10.222.0.39 () {56 vars in 1139 bytes} [Mon Aug 8 02:13:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:40.988337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1f44e-af99-484d-9595-1ed93427e93d None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:40.989143 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1f44e-af99-484d-9595-1ed93427e93d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:40.989461 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1f44e-af99-484d-9595-1ed93427e93d None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:40.994293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 333/665] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:40.994293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1f44e-af99-484d-9595-1ed93427e93d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:41.025325 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:41.081898 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3e00f-d497-4b6f-9663-7c4a5e96c3d1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:41.105292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1f44e-af99-484d-9595-1ed93427e93d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:41.117504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52d893-e08e-4349-a3e7-2e5e01c670f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:41.118176 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52d893-e08e-4349-a3e7-2e5e01c670f0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:41.118490 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52d893-e08e-4349-a3e7-2e5e01c670f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:41.118742 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52d893-e08e-4349-a3e7-2e5e01c670f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:41.125901 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc52d893-e08e-4349-a3e7-2e5e01c670f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:41.127407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc52d893-e08e-4349-a3e7-2e5e01c670f0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:41.136226 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:41.138848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-bc52d893-e08e-4349-a3e7-2e5e01c670f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:41.138848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:41.138848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:41.138848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:41.138848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:41.138848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:41.138848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:41.138848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:41.138848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:41.138848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:41.138848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:41.138848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:41.138848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:41.138848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:41.138848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:41.139704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:41.139704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:41.139704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:41.139704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:41.139704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:41.139704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:41.139704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:41.139704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:41.139704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:41.139704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:41.139704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:41.139704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:41.139704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:41.139704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:41.139704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:41.139704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:41.139704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:41.140566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:41.140566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:41.140566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:41.140566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:41.140566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:41.140566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:41.140566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:41.140566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:41.142393 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 333/666] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:41.143183 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52d893-e08e-4349-a3e7-2e5e01c670f0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:41.147099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03738ca1-1365-4da7-8e47-a0dcc2be3b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:41.147734 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03738ca1-1365-4da7-8e47-a0dcc2be3b96 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:41.148018 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03738ca1-1365-4da7-8e47-a0dcc2be3b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:41.148293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03738ca1-1365-4da7-8e47-a0dcc2be3b96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:41.155236 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03738ca1-1365-4da7-8e47-a0dcc2be3b96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:41.156679 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03738ca1-1365-4da7-8e47-a0dcc2be3b96 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:41.168577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-03738ca1-1365-4da7-8e47-a0dcc2be3b96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:41.168577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:41.168577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:41.168577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:41.168577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:41.168577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:41.168577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:41.168577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:41.168577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:41.168577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:41.168577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:41.168577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:41.168577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:41.168577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:41.168577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:41.169440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:41.169440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:41.169440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:41.169440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:41.169440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:41.169440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:41.169440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:41.169440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:41.169440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:41.169440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:41.169440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:41.169440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:41.169440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:41.169440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:41.169440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:41.169440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:41.169440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:41.170885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:41.170885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:41.170885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:41.170885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:41.170885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:41.170885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:41.170885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:41.170885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:41.170885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 334/667] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:41.171382 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d895e-2627-4b4b-8f8c-7ba584428879 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:41.171450 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d895e-2627-4b4b-8f8c-7ba584428879 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:41.171695 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d895e-2627-4b4b-8f8c-7ba584428879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:41.190064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b84d895e-2627-4b4b-8f8c-7ba584428879 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:41.191229 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b84d895e-2627-4b4b-8f8c-7ba584428879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:41.250632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52d893-e08e-4349-a3e7-2e5e01c670f0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:41.253041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:41.274542 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 335/668] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:41.275330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d895e-2627-4b4b-8f8c-7ba584428879 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:41.285111 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ca257-1496-4c15-892a-ef17067f4560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:41.286224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ca257-1496-4c15-892a-ef17067f4560 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:41.286537 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ca257-1496-4c15-892a-ef17067f4560 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:41.286801 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ca257-1496-4c15-892a-ef17067f4560 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:41.368617 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ca257-1496-4c15-892a-ef17067f4560 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:41.370073 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ca257-1496-4c15-892a-ef17067f4560 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:41.386641 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d895e-2627-4b4b-8f8c-7ba584428879 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:41.409325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:41.425701 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021a313-56ba-4337-808e-d587072789db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:41.426383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021a313-56ba-4337-808e-d587072789db None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:41.426669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021a313-56ba-4337-808e-d587072789db None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:41.426940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021a313-56ba-4337-808e-d587072789db None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:41.427628 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021a313-56ba-4337-808e-d587072789db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:41.427933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021a313-56ba-4337-808e-d587072789db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:41.429350 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021a313-56ba-4337-808e-d587072789db None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:41.441445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 336/669] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:41.446882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021a313-56ba-4337-808e-d587072789db None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:41.487237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 334/670] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:41.488454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ca257-1496-4c15-892a-ef17067f4560 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:41.493293 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:41.553450 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021a313-56ba-4337-808e-d587072789db None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:41.588783 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be978f1e-ee3b-4265-bd2e-983c49e5dd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:41.589377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be978f1e-ee3b-4265-bd2e-983c49e5dd64 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:41.589654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be978f1e-ee3b-4265-bd2e-983c49e5dd64 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:41.589925 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be978f1e-ee3b-4265-bd2e-983c49e5dd64 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:41.590660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be978f1e-ee3b-4265-bd2e-983c49e5dd64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:41.590967 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be978f1e-ee3b-4265-bd2e-983c49e5dd64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:41.592481 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be978f1e-ee3b-4265-bd2e-983c49e5dd64 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:41.597537 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ca257-1496-4c15-892a-ef17067f4560 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:41.606007 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 337/671] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:41.611489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be978f1e-ee3b-4265-bd2e-983c49e5dd64 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:41.633640 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ba36a-ed6f-486e-8c31-5203e12c7cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:41.634703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ba36a-ed6f-486e-8c31-5203e12c7cef None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:41.634982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ba36a-ed6f-486e-8c31-5203e12c7cef None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:41.635404 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ba36a-ed6f-486e-8c31-5203e12c7cef None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:41.646359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ba36a-ed6f-486e-8c31-5203e12c7cef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:41.650985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-178ba36a-ed6f-486e-8c31-5203e12c7cef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:41.650985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:41.650985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:41.650985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:41.650985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:41.650985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:41.650985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:41.650985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:41.650985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:41.650985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:41.650985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:41.650985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:41.650985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:41.650985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:41.650985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:41.650985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:41.651982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:41.651982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:41.651982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:41.651982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:41.651982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:41.651982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:41.651982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:41.651982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:41.651982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:41.651982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:41.651982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:41.651982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:41.651982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:41.651982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:41.651982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:41.651982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:41.658505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:41.658505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:41.658505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:41.658505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:41.658505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:41.658505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:41.658505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:41.658505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:41.658505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:41.658505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:41.658505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:41.658505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:41.658505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 335/672] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:41.658505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ba36a-ed6f-486e-8c31-5203e12c7cef None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:41.661448 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:41.724947 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:41.729440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be978f1e-ee3b-4265-bd2e-983c49e5dd64 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:41.763607 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8999ba5-8bee-48c6-bddb-06e8a7d5b992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:41.764204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8999ba5-8bee-48c6-bddb-06e8a7d5b992 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:41.764488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8999ba5-8bee-48c6-bddb-06e8a7d5b992 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:41.764763 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8999ba5-8bee-48c6-bddb-06e8a7d5b992 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:41.765480 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8999ba5-8bee-48c6-bddb-06e8a7d5b992 None admin] RBAC: Adding query filter params (role.id=838b081e674f4c0ab4a33bcf8e614dc5, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=7776a6a1faf34b369e399d83bb3e18c7) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:41.765790 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8999ba5-8bee-48c6-bddb-06e8a7d5b992 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:41.765943 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ba36a-ed6f-486e-8c31-5203e12c7cef None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:41.767248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8999ba5-8bee-48c6-bddb-06e8a7d5b992 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:41.773484 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 338/673] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:41 2022] GET /identity/v3/role_assignments?user.id=7776a6a1faf34b369e399d83bb3e18c7&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=838b081e674f4c0ab4a33bcf8e614dc5 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:41.774286 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8999ba5-8bee-48c6-bddb-06e8a7d5b992 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:41.801807 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8673d-fa90-4a40-8d85-489854ab5759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:41.802478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8673d-fa90-4a40-8d85-489854ab5759 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:41.802761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8673d-fa90-4a40-8d85-489854ab5759 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:41.803026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8673d-fa90-4a40-8d85-489854ab5759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:41.814353 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8673d-fa90-4a40-8d85-489854ab5759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:41.815859 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8673d-fa90-4a40-8d85-489854ab5759 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:41.832039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e5e8673d-fa90-4a40-8d85-489854ab5759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:41.832039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:41.832039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:41.832039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:41.832039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:41.832039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:41.832039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:41.832039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:41.832039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:41.832039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:41.832039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:41.832039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:41.832039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:41.832039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:41.832039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:41.832039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:41.833704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:41.833704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:41.833704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:41.833704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:41.833704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:41.833704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:41.833704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:41.833704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:41.833704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:41.833704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:41.833704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:41.833704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:41.833704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:41.833704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:41.833704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:41.833704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:41.833704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:41.834615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:41.834615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:41.834615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:41.834615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 336/674] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:41.835208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8673d-fa90-4a40-8d85-489854ab5759 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:41.900324 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8999ba5-8bee-48c6-bddb-06e8a7d5b992 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:41.902794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:41.941487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba31a3-0dde-4777-a19a-4dd5e390e8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:41.942662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba31a3-0dde-4777-a19a-4dd5e390e8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:41.942939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba31a3-0dde-4777-a19a-4dd5e390e8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:41.943213 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba31a3-0dde-4777-a19a-4dd5e390e8c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:41.943942 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba31a3-0dde-4777-a19a-4dd5e390e8c3 None admin] RBAC: Adding query filter params (role.id=838b081e674f4c0ab4a33bcf8e614dc5, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=2812e59d0d9948879a465fe3b5b421e1) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:41.944249 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba31a3-0dde-4777-a19a-4dd5e390e8c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:41.946299 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba31a3-0dde-4777-a19a-4dd5e390e8c3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:41.951183 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8673d-fa90-4a40-8d85-489854ab5759 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:41.951898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 339/675] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:41 2022] GET /identity/v3/role_assignments?user.id=2812e59d0d9948879a465fe3b5b421e1&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=838b081e674f4c0ab4a33bcf8e614dc5 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:41.952875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccbf65-b30f-40c4-8438-6da848f8d379 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:41.953157 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccbf65-b30f-40c4-8438-6da848f8d379 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:41.953433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccbf65-b30f-40c4-8438-6da848f8d379 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:41.954695 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 340/676] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:41.955384 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccbf65-b30f-40c4-8438-6da848f8d379 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:41.977642 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:42.003333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad95fb0-378e-4219-813c-6b7f2baac4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:42.003959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad95fb0-378e-4219-813c-6b7f2baac4af None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:42.004781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad95fb0-378e-4219-813c-6b7f2baac4af None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:42.005042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad95fb0-378e-4219-813c-6b7f2baac4af None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:42.005767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad95fb0-378e-4219-813c-6b7f2baac4af None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:42.006065 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad95fb0-378e-4219-813c-6b7f2baac4af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:42.007844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad95fb0-378e-4219-813c-6b7f2baac4af None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:42.021913 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 337/677] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:13:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:42.023941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee363dd-4888-48c4-9958-a5768aed3860 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:42.024592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee363dd-4888-48c4-9958-a5768aed3860 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:42.025553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee363dd-4888-48c4-9958-a5768aed3860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:42.048199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2ee363dd-4888-48c4-9958-a5768aed3860 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:42.050751 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2ee363dd-4888-48c4-9958-a5768aed3860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:42.099386 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccbf65-b30f-40c4-8438-6da848f8d379 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:42.131078 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 338/678] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:42.132290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee363dd-4888-48c4-9958-a5768aed3860 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:42.153026 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb776e-a337-4fc7-98bc-bd9a5f29bfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:42.153731 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb776e-a337-4fc7-98bc-bd9a5f29bfe5 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:42.154017 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb776e-a337-4fc7-98bc-bd9a5f29bfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:42.154310 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb776e-a337-4fc7-98bc-bd9a5f29bfe5 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:42.155013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb776e-a337-4fc7-98bc-bd9a5f29bfe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:42.155318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb776e-a337-4fc7-98bc-bd9a5f29bfe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:42.156935 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb776e-a337-4fc7-98bc-bd9a5f29bfe5 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:42.171693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 341/679] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:42.173351 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb776e-a337-4fc7-98bc-bd9a5f29bfe5 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:42.211597 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:42.259569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee363dd-4888-48c4-9958-a5768aed3860 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:42.270388 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:42.297502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d522fa-9461-45f0-80e5-9f0d342a784f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:42.298145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d522fa-9461-45f0-80e5-9f0d342a784f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:42.298458 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d522fa-9461-45f0-80e5-9f0d342a784f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:42.298793 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d522fa-9461-45f0-80e5-9f0d342a784f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:42.299581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb776e-a337-4fc7-98bc-bd9a5f29bfe5 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:42.310222 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d522fa-9461-45f0-80e5-9f0d342a784f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:42.311722 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d522fa-9461-45f0-80e5-9f0d342a784f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:42.331053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-59d522fa-9461-45f0-80e5-9f0d342a784f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:42.331053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:42.331053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:42.331053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:42.331053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:42.331053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:42.331053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:42.331053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:42.331053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:42.331053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:42.331053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:42.331053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:42.331053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:42.331053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:42.331053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:42.331053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:42.332051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:42.332051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:42.332051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:42.332051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:42.332051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:42.332051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:42.332051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:42.332051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:42.332051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:42.332051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:42.332051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:42.332051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:42.332051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:42.332051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:42.332051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:42.332051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:42.332051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:42.332992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:42.332992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:42.332992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:42.332992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 339/680] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:42.333219 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d522fa-9461-45f0-80e5-9f0d342a784f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:42.344139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3287a4-51db-487d-87bd-f1fc3b4e36ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:42.345117 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3287a4-51db-487d-87bd-f1fc3b4e36ed None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:42.345414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3287a4-51db-487d-87bd-f1fc3b4e36ed None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:42.345689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3287a4-51db-487d-87bd-f1fc3b4e36ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:42.359465 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3287a4-51db-487d-87bd-f1fc3b4e36ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:42.366300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-8b3287a4-51db-487d-87bd-f1fc3b4e36ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:42.366300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:42.366300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:42.366300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:42.366300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:42.366300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:42.366300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:42.366300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:42.366300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:42.366300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:42.366300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:42.366300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:42.366300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:42.366300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:42.366300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:42.366300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:42.368023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:42.368023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:42.368023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:42.368023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:42.368023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:42.368023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:42.368023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:42.368023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:42.368023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:42.368023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:42.368023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:42.368023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:42.368023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:42.368023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:42.368023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:42.368023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:42.368994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:42.368994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:42.368994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:42.368994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:42.368994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:42.368994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:42.368994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:42.368994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:42.368994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:42.368994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:42.368994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:42.368994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:42.368994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 342/681] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:42.368994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3287a4-51db-487d-87bd-f1fc3b4e36ed None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:42.459214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d522fa-9461-45f0-80e5-9f0d342a784f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:42.477780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:42.494916 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3287a4-51db-487d-87bd-f1fc3b4e36ed None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:42.499225 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf60d2-e539-4fb6-a1f2-7c02956fa43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:42.499833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf60d2-e539-4fb6-a1f2-7c02956fa43c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:42.500111 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf60d2-e539-4fb6-a1f2-7c02956fa43c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:42.500379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf60d2-e539-4fb6-a1f2-7c02956fa43c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:42.508063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf60d2-e539-4fb6-a1f2-7c02956fa43c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:42.509680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf60d2-e539-4fb6-a1f2-7c02956fa43c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:42.521020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-efdf60d2-e539-4fb6-a1f2-7c02956fa43c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:42.521020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:42.521020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:42.521020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:42.521020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:42.521020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:42.521020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:42.521020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:42.521020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:42.521020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:42.521020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:42.521020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:42.521020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:42.521020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:42.521020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:42.522136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:42.522136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:42.522136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:42.522136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:42.522136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:42.522136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:42.522136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:42.522136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:42.522136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:42.522136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:42.522136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:42.522136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:42.522136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:42.522136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:42.522136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:42.522136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:42.522136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:42.523165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:42.523165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:42.523165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:42.523165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:42.523165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:42.523165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:42.523165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:42.523165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:42.523165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 340/682] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:42.523661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf60d2-e539-4fb6-a1f2-7c02956fa43c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:42.538319 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884c4da-ce86-4891-8488-7166b6914463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:42.539153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884c4da-ce86-4891-8488-7166b6914463 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:42.539666 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884c4da-ce86-4891-8488-7166b6914463 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:42.540091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884c4da-ce86-4891-8488-7166b6914463 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:42.540885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884c4da-ce86-4891-8488-7166b6914463 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:42.541339 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884c4da-ce86-4891-8488-7166b6914463 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:42.542892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884c4da-ce86-4891-8488-7166b6914463 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:42.550797 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:42.556532 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 343/683] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:42.557671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884c4da-ce86-4891-8488-7166b6914463 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:42.641233 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf60d2-e539-4fb6-a1f2-7c02956fa43c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:42.672802 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884c4da-ce86-4891-8488-7166b6914463 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:42.677828 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c178b-1281-4f3c-b74a-73f0073f4d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:42.678470 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c178b-1281-4f3c-b74a-73f0073f4d16 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:42.678754 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c178b-1281-4f3c-b74a-73f0073f4d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:42.679026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c178b-1281-4f3c-b74a-73f0073f4d16 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:42.679688 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c178b-1281-4f3c-b74a-73f0073f4d16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:42.679990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c178b-1281-4f3c-b74a-73f0073f4d16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:42.681421 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c178b-1281-4f3c-b74a-73f0073f4d16 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:42.694476 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 341/684] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:42.694476 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c178b-1281-4f3c-b74a-73f0073f4d16 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:42.716939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5b9b9-6717-417a-b7ae-8ed92f2e4623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:42.717547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5b9b9-6717-417a-b7ae-8ed92f2e4623 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:42.717827 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5b9b9-6717-417a-b7ae-8ed92f2e4623 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:42.718115 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5b9b9-6717-417a-b7ae-8ed92f2e4623 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:42.718843 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5b9b9-6717-417a-b7ae-8ed92f2e4623 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:42.719153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5b9b9-6717-417a-b7ae-8ed92f2e4623 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:42.720752 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5b9b9-6717-417a-b7ae-8ed92f2e4623 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:42.738076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 344/685] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:42.739739 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5b9b9-6717-417a-b7ae-8ed92f2e4623 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:42.757466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:42.794291 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:42.817717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c178b-1281-4f3c-b74a-73f0073f4d16 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:42.843206 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5b9b9-6717-417a-b7ae-8ed92f2e4623 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:42.862630 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9ad92-cbab-485f-8b4e-abce3596879b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:42.863517 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9ad92-cbab-485f-8b4e-abce3596879b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:42.864577 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9ad92-cbab-485f-8b4e-abce3596879b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:42.864995 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9ad92-cbab-485f-8b4e-abce3596879b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:42.875678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9ad92-cbab-485f-8b4e-abce3596879b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:42.877597 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9ad92-cbab-485f-8b4e-abce3596879b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:42.880167 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2218d33-0b68-4713-b050-4a39a572c7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:42.880907 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2218d33-0b68-4713-b050-4a39a572c7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:42.881297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2218d33-0b68-4713-b050-4a39a572c7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:42.881682 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2218d33-0b68-4713-b050-4a39a572c7e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:42.891350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-cdf9ad92-cbab-485f-8b4e-abce3596879b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:13:42.891350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:42.891350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:42.891350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:42.891350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:42.891350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:42.891350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:42.891350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:42.891350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:42.891350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:42.891350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:42.891350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:42.891350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:42.891350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:42.891350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:42.891350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:42.892442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:42.892442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:42.892442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:42.892442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:42.892442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:42.892442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:42.892442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:42.892442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:42.892442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:42.892442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:42.892442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:42.892442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:42.892442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:42.892442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:42.892442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:42.892442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:42.892442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:42.892442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:42.893601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:42.893601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:42.893601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:42.893601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:42.893601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:42.893601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:42.893601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:42.893601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:42.893601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:13:42.893601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:42.900610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 342/686] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:42.900610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9ad92-cbab-485f-8b4e-abce3596879b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:42.900830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2218d33-0b68-4713-b050-4a39a572c7e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:42.900830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2218d33-0b68-4713-b050-4a39a572c7e2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:42.920762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-e2218d33-0b68-4713-b050-4a39a572c7e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:42.920762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:42.920762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:42.920762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:42.920762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:42.920762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:42.920762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:42.920762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:42.920762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:42.920762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:42.920762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:42.920762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:42.920762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:42.920762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:42.920762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:42.920762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:42.921710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:42.921710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:42.921710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:42.921710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:42.921710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:42.921710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:42.921710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:42.921710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:42.921710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:42.921710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:42.921710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:42.921710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:42.921710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:42.921710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:42.921710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:42.921710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:42.921710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:42.922609 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:42.922609 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:42.922609 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:42.925366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 345/687] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:42.926533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2218d33-0b68-4713-b050-4a39a572c7e2 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.002732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:43.010866 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9ad92-cbab-485f-8b4e-abce3596879b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:43.053110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89e5b1-4192-4552-8919-24ca4aaf016c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:43.053751 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89e5b1-4192-4552-8919-24ca4aaf016c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.054101 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89e5b1-4192-4552-8919-24ca4aaf016c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.054472 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89e5b1-4192-4552-8919-24ca4aaf016c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.055271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89e5b1-4192-4552-8919-24ca4aaf016c None admin] RBAC: Adding query filter params (role.id=838b081e674f4c0ab4a33bcf8e614dc5, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=78506fea1c024b169a4b7ee86116658c) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:43.055658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89e5b1-4192-4552-8919-24ca4aaf016c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:43.057186 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89e5b1-4192-4552-8919-24ca4aaf016c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:43.057305 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2218d33-0b68-4713-b050-4a39a572c7e2 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:43.070664 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 343/688] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:42 2022] GET /identity/v3/role_assignments?user.id=78506fea1c024b169a4b7ee86116658c&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=838b081e674f4c0ab4a33bcf8e614dc5 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:43.071833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89e5b1-4192-4552-8919-24ca4aaf016c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.088169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:43.106314 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5a91c-d5c0-4827-9412-acd1dd876a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:43.106314 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5a91c-d5c0-4827-9412-acd1dd876a44 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.106314 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5a91c-d5c0-4827-9412-acd1dd876a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.106314 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5a91c-d5c0-4827-9412-acd1dd876a44 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.106773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5a91c-d5c0-4827-9412-acd1dd876a44 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:43.106773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5a91c-d5c0-4827-9412-acd1dd876a44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:43.106773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5a91c-d5c0-4827-9412-acd1dd876a44 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:43.126400 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 346/689] 10.222.0.39 () {56 vars in 1135 bytes} [Mon Aug 8 02:13:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:43.138095 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5a91c-d5c0-4827-9412-acd1dd876a44 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.232911 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89e5b1-4192-4552-8919-24ca4aaf016c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:43.271981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae19f7-47fe-42fd-8b0a-8fb8951ec05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:43.272584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae19f7-47fe-42fd-8b0a-8fb8951ec05a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.272862 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae19f7-47fe-42fd-8b0a-8fb8951ec05a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.273130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae19f7-47fe-42fd-8b0a-8fb8951ec05a None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.273769 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae19f7-47fe-42fd-8b0a-8fb8951ec05a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:43.274072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae19f7-47fe-42fd-8b0a-8fb8951ec05a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:43.275583 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae19f7-47fe-42fd-8b0a-8fb8951ec05a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:43.294280 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 344/690] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:43.296409 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae19f7-47fe-42fd-8b0a-8fb8951ec05a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.317528 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5a91c-d5c0-4827-9412-acd1dd876a44 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:43.339459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:43.361608 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb003766-78f7-4176-bf84-4825160e7e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:43.362569 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb003766-78f7-4176-bf84-4825160e7e39 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.363243 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb003766-78f7-4176-bf84-4825160e7e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.363700 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb003766-78f7-4176-bf84-4825160e7e39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.387729 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:43.437097 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae19f7-47fe-42fd-8b0a-8fb8951ec05a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:43.465303 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb003766-78f7-4176-bf84-4825160e7e39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:43.466838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb003766-78f7-4176-bf84-4825160e7e39 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:43.474449 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0965f1-7558-4548-841a-5f5899b1a32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:43.475266 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0965f1-7558-4548-841a-5f5899b1a32c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.475825 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0965f1-7558-4548-841a-5f5899b1a32c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.476229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0965f1-7558-4548-841a-5f5899b1a32c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.483955 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0965f1-7558-4548-841a-5f5899b1a32c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:43.485558 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0965f1-7558-4548-841a-5f5899b1a32c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:43.497468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-4b0965f1-7558-4548-841a-5f5899b1a32c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:13:43.497468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:43.497468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:43.497468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:43.497468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:43.497468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:43.497468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:43.497468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:43.497468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:43.497468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:43.497468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:43.497468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:43.497468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:43.497468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:43.497468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:43.497468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:43.498626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:43.498626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:43.498626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:43.498626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:43.498626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:43.498626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:43.498626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:43.498626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:43.498626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:43.498626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:43.498626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:43.498626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:43.498626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:43.498626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:43.498626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:43.498626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:43.498626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:43.498626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:43.499543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:43.499543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:43.499543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:43.499543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:43.499543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:43.499543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:43.499543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:43.499543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:43.499543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:13:43.499543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:43.501643 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 345/691] 10.222.0.39 () {56 vars in 1093 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:43.516428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0965f1-7558-4548-841a-5f5899b1a32c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.595207 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 347/692] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:43.602391 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb003766-78f7-4176-bf84-4825160e7e39 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.634857 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:43.712113 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0965f1-7558-4548-841a-5f5899b1a32c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:43.766532 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:43.783314 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da50f1b-6442-4917-9f03-73f851b25389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:43.783921 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da50f1b-6442-4917-9f03-73f851b25389 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.784203 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da50f1b-6442-4917-9f03-73f851b25389 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.784476 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da50f1b-6442-4917-9f03-73f851b25389 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.785213 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da50f1b-6442-4917-9f03-73f851b25389 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:43.785553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da50f1b-6442-4917-9f03-73f851b25389 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:43.787021 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da50f1b-6442-4917-9f03-73f851b25389 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:43.793941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb003766-78f7-4176-bf84-4825160e7e39 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:43.827617 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 346/693] 10.222.0.39 () {56 vars in 1133 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:43.832901 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da50f1b-6442-4917-9f03-73f851b25389 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.862783 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef01c07-9343-4d74-8448-33835352d8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:43.864339 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef01c07-9343-4d74-8448-33835352d8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.864877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef01c07-9343-4d74-8448-33835352d8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.865361 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef01c07-9343-4d74-8448-33835352d8c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.880660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef01c07-9343-4d74-8448-33835352d8c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:43.882486 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef01c07-9343-4d74-8448-33835352d8c4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:43.897725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-bef01c07-9343-4d74-8448-33835352d8c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:43.897725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:43.897725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:43.897725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:43.897725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:43.897725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:43.897725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:43.897725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:43.897725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:43.897725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:43.897725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:43.897725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:43.897725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:43.897725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:43.897725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:43.897725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:43.898824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:43.898824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:43.898824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:43.898824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:43.898824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:43.898824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:43.898824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:43.898824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:43.898824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:43.898824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:43.898824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:43.898824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:43.898824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:43.898824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:43.898824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:43.898824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:43.898824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:43.899766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:43.899766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:43.899766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:43.901891 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 348/694] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:43.907533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef01c07-9343-4d74-8448-33835352d8c4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.985619 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:44.011798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef01c07-9343-4d74-8448-33835352d8c4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:44.047062 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc16a36-38a9-4dc7-a931-812ced4a1f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:44.047835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc16a36-38a9-4dc7-a931-812ced4a1f66 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.048212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc16a36-38a9-4dc7-a931-812ced4a1f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.048585 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc16a36-38a9-4dc7-a931-812ced4a1f66 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.049335 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc16a36-38a9-4dc7-a931-812ced4a1f66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:44.049628 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da50f1b-6442-4917-9f03-73f851b25389 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:44.049818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc16a36-38a9-4dc7-a931-812ced4a1f66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:44.051865 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc16a36-38a9-4dc7-a931-812ced4a1f66 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:44.064598 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 349/695] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:44.070094 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc16a36-38a9-4dc7-a931-812ced4a1f66 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:44.110051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa610781-48de-4902-8ac5-101b399542b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:44.110739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa610781-48de-4902-8ac5-101b399542b0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.111041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa610781-48de-4902-8ac5-101b399542b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.111316 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa610781-48de-4902-8ac5-101b399542b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.175119 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc16a36-38a9-4dc7-a931-812ced4a1f66 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:44.198152 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:44.208460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:44.219501 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa610781-48de-4902-8ac5-101b399542b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:44.219641 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8114f2e-3718-4c01-87a4-cf2a4627648f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:44.220172 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8114f2e-3718-4c01-87a4-cf2a4627648f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.220457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8114f2e-3718-4c01-87a4-cf2a4627648f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.220728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8114f2e-3718-4c01-87a4-cf2a4627648f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.222020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa610781-48de-4902-8ac5-101b399542b0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:44.228725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8114f2e-3718-4c01-87a4-cf2a4627648f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:44.230285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8114f2e-3718-4c01-87a4-cf2a4627648f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:44.243260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-a8114f2e-3718-4c01-87a4-cf2a4627648f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:44.243260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:44.243260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:44.243260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:44.243260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:44.243260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:44.243260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:44.243260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:44.243260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:44.243260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:44.243260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:44.243260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:44.243260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:44.243260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:44.243260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:44.244158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:44.244158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:44.244158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:44.244158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:44.244158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:44.244158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:44.244158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:44.244158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:44.244158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:44.244158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:44.244158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:44.244158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:44.244158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:44.244158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:44.244158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:44.244158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:44.244158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:44.245405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:44.245405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:44.245405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:44.245405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:44.245405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:44.245405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:44.245405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:44.245405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:44.245405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 350/696] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:44.245898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4a7a0-f5fb-4f95-9dad-1c8717090469 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.246192 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4a7a0-f5fb-4f95-9dad-1c8717090469 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.251381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4a7a0-f5fb-4f95-9dad-1c8717090469 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.256829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 351/697] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:44.256829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4a7a0-f5fb-4f95-9dad-1c8717090469 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:44.343813 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 347/698] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:44.343813 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66cb26-31d0-4700-b253-41b6e9d6d08a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.343813 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66cb26-31d0-4700-b253-41b6e9d6d08a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.343813 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66cb26-31d0-4700-b253-41b6e9d6d08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.359380 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fb66cb26-31d0-4700-b253-41b6e9d6d08a None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:44.360565 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fb66cb26-31d0-4700-b253-41b6e9d6d08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:44.372877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4a7a0-f5fb-4f95-9dad-1c8717090469 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:44.408274 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778964f-4722-4547-b942-daa5d26098d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:44.408977 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778964f-4722-4547-b942-daa5d26098d9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.409256 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778964f-4722-4547-b942-daa5d26098d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.409522 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778964f-4722-4547-b942-daa5d26098d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.410199 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778964f-4722-4547-b942-daa5d26098d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:44.411082 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778964f-4722-4547-b942-daa5d26098d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:44.412632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778964f-4722-4547-b942-daa5d26098d9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:44.429313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 352/699] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:44.429313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de4dc2-686b-4fa3-b350-4fc1a0deaa35 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.429313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de4dc2-686b-4fa3-b350-4fc1a0deaa35 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.429313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de4dc2-686b-4fa3-b350-4fc1a0deaa35 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.429313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 353/700] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:44.429313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de4dc2-686b-4fa3-b350-4fc1a0deaa35 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:44.437407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 348/701] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:44.443248 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66cb26-31d0-4700-b253-41b6e9d6d08a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:44.466160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:44.494519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:44.551291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de4dc2-686b-4fa3-b350-4fc1a0deaa35 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:44.562008 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66cb26-31d0-4700-b253-41b6e9d6d08a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:44.588197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b0b73-daee-4782-8297-428e0638d3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:44.588862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b0b73-daee-4782-8297-428e0638d3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.589144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b0b73-daee-4782-8297-428e0638d3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.589412 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b0b73-daee-4782-8297-428e0638d3e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.597462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf96b88-e72b-40c8-b88d-25ad2f2bede0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:44.598678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf96b88-e72b-40c8-b88d-25ad2f2bede0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.598957 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf96b88-e72b-40c8-b88d-25ad2f2bede0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.599234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf96b88-e72b-40c8-b88d-25ad2f2bede0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.599957 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf96b88-e72b-40c8-b88d-25ad2f2bede0 None admin] RBAC: Adding query filter params (role.id=838b081e674f4c0ab4a33bcf8e614dc5, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=ac60b4d621234225830c94edfac400e2) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:44.600267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf96b88-e72b-40c8-b88d-25ad2f2bede0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:44.601588 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b0b73-daee-4782-8297-428e0638d3e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:44.601701 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf96b88-e72b-40c8-b88d-25ad2f2bede0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:44.603322 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b0b73-daee-4782-8297-428e0638d3e2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:44.611646 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 349/702] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:44 2022] GET /identity/v3/role_assignments?user.id=ac60b4d621234225830c94edfac400e2&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=838b081e674f4c0ab4a33bcf8e614dc5 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:44.613106 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0adb98-ff80-4b9f-8c55-7d11b5c2533b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.613777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0adb98-ff80-4b9f-8c55-7d11b5c2533b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.614272 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0adb98-ff80-4b9f-8c55-7d11b5c2533b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.633679 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1e0adb98-ff80-4b9f-8c55-7d11b5c2533b None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:44.635341 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1e0adb98-ff80-4b9f-8c55-7d11b5c2533b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:44.636789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-428b0b73-daee-4782-8297-428e0638d3e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:44.636789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:44.636789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:44.636789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:44.636789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:44.636789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:44.636789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:44.636789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:44.636789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:44.636789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:44.636789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:44.636789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:44.636789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:44.636789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:44.636789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:44.636789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:44.637738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:44.637738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:44.637738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:44.637738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:44.637738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:44.637738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:44.637738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:44.637738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:44.637738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:44.637738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:44.637738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:44.637738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:44.637738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:44.637738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:44.637738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:44.637738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:44.637738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:44.638706 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:44.638706 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:44.638706 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:44.638706 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 354/703] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:44.639336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b0b73-daee-4782-8297-428e0638d3e2 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:44.715236 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 350/704] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:44.716187 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0adb98-ff80-4b9f-8c55-7d11b5c2533b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:44.742628 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:44.789523 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b0b73-daee-4782-8297-428e0638d3e2 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:44.791113 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:44.833977 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b6abe-ec64-4f5b-bc4f-4952cc7ab327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:44.834659 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b6abe-ec64-4f5b-bc4f-4952cc7ab327 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.834955 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b6abe-ec64-4f5b-bc4f-4952cc7ab327 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.835157 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b6abe-ec64-4f5b-bc4f-4952cc7ab327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.846160 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b6abe-ec64-4f5b-bc4f-4952cc7ab327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:44.847692 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b6abe-ec64-4f5b-bc4f-4952cc7ab327 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:44.857490 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0adb98-ff80-4b9f-8c55-7d11b5c2533b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:44.863478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-482b6abe-ec64-4f5b-bc4f-4952cc7ab327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:44.863478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:44.863478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:44.863478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:44.863478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:44.863478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:44.863478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:44.863478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:44.863478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:44.863478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:44.863478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:44.863478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:44.863478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:44.863478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:44.863478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:44.863478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:44.864477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:44.864477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:44.864477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:44.864477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:44.864477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:44.864477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:44.864477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:44.864477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:44.864477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:44.864477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:44.864477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:44.864477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:44.864477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:44.864477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:44.864477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:44.864477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:44.864477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:44.865413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:44.865413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:44.865413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:44.867197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 355/705] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:44.868211 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b6abe-ec64-4f5b-bc4f-4952cc7ab327 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:44.906365 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8435f-4ef8-46d3-ac2a-c14bf3c48d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:44.907311 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8435f-4ef8-46d3-ac2a-c14bf3c48d23 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.907592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8435f-4ef8-46d3-ac2a-c14bf3c48d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.907863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8435f-4ef8-46d3-ac2a-c14bf3c48d23 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.908511 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8435f-4ef8-46d3-ac2a-c14bf3c48d23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:44.908817 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8435f-4ef8-46d3-ac2a-c14bf3c48d23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:44.910197 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8435f-4ef8-46d3-ac2a-c14bf3c48d23 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:44.925522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 351/706] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:44.926613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8435f-4ef8-46d3-ac2a-c14bf3c48d23 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:45.010714 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:45.012638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b6abe-ec64-4f5b-bc4f-4952cc7ab327 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:45.053043 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da77257-0295-4878-80b6-beb966fad003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:45.053756 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8435f-4ef8-46d3-ac2a-c14bf3c48d23 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:45.054195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da77257-0295-4878-80b6-beb966fad003 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:45.054697 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da77257-0295-4878-80b6-beb966fad003 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:45.055148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da77257-0295-4878-80b6-beb966fad003 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:45.055910 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da77257-0295-4878-80b6-beb966fad003 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:45.057461 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da77257-0295-4878-80b6-beb966fad003 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:45.070771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-8da77257-0295-4878-80b6-beb966fad003 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:13:45.070771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:45.070771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:13:45.070771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:13:45.070771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:13:45.070771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:13:45.070771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:13:45.070771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:13:45.070771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:13:45.070771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:45.070771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:13:45.070771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:45.070771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:45.070771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:45.070771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:45.070771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:45.070771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:45.071846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:45.071846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:45.071846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:45.071846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:45.071846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:45.071846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:45.071846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:13:45.071846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:13:45.071846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:13:45.071846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:13:45.071846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:45.071846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:45.071846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:45.071846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:45.071846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:45.071846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:45.072718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:45.072718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:45.072718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:45.072718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:45.072718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:45.072718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:45.072718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:45.072718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:45.072718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:45.072718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:45.072718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:13:45.072718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:13:45.072718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:13:45.072718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:45.075475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 356/707] 10.222.0.39 () {56 vars in 1129 bytes} [Mon Aug 8 02:13:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:45.076491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da77257-0295-4878-80b6-beb966fad003 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:45.100288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:45.102350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89216fbc-a011-4bfa-8663-087b6b678280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:45.102959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89216fbc-a011-4bfa-8663-087b6b678280 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:45.104505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89216fbc-a011-4bfa-8663-087b6b678280 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:45.104505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89216fbc-a011-4bfa-8663-087b6b678280 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:45.104771 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89216fbc-a011-4bfa-8663-087b6b678280 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:45.105041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89216fbc-a011-4bfa-8663-087b6b678280 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:45.106757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89216fbc-a011-4bfa-8663-087b6b678280 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:45.126335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 352/708] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:45.127549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89216fbc-a011-4bfa-8663-087b6b678280 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:45.198115 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da77257-0295-4878-80b6-beb966fad003 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:45.250125 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44371733-15f8-449e-abe0-6589e75bace2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:45.251147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44371733-15f8-449e-abe0-6589e75bace2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:45.251635 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44371733-15f8-449e-abe0-6589e75bace2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:45.252188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44371733-15f8-449e-abe0-6589e75bace2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:45.259853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89216fbc-a011-4bfa-8663-087b6b678280 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:45.261932 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44371733-15f8-449e-abe0-6589e75bace2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:45.263700 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44371733-15f8-449e-abe0-6589e75bace2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:45.275794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-44371733-15f8-449e-abe0-6589e75bace2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:45.275794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:45.275794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:45.275794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:45.275794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:45.275794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:45.275794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:45.275794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:45.275794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:45.275794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:45.275794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:45.275794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:45.275794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:45.275794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:45.275794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:45.276661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:45.276661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:45.276661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:45.276661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:45.276661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:45.276661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:45.276661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:45.276661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:45.276661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:45.276661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:45.276661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:45.276661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:45.276661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:45.276661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:45.276661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:45.276661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:45.276661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:45.277533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:45.277533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:45.277533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:45.277533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:45.277533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:45.277533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:45.277533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:45.277533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:45.280021 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 357/709] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:45.280969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44371733-15f8-449e-abe0-6589e75bace2 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:45.289461 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:45.299560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458151e-cb50-42da-9d78-5deda4869340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:45.300487 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458151e-cb50-42da-9d78-5deda4869340 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:45.300773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458151e-cb50-42da-9d78-5deda4869340 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:45.301591 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458151e-cb50-42da-9d78-5deda4869340 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:45.302770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458151e-cb50-42da-9d78-5deda4869340 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:45.303142 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458151e-cb50-42da-9d78-5deda4869340 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:45.304871 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458151e-cb50-42da-9d78-5deda4869340 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:45.311612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 353/710] 10.222.0.39 () {56 vars in 1124 bytes} [Mon Aug 8 02:13:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:45.312631 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458151e-cb50-42da-9d78-5deda4869340 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:45.392704 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:45.423011 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44371733-15f8-449e-abe0-6589e75bace2 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:45.427442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458151e-cb50-42da-9d78-5deda4869340 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:45.468290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9081a-baf1-432f-9452-1cd64b7f1721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:45.468896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9081a-baf1-432f-9452-1cd64b7f1721 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:45.469177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9081a-baf1-432f-9452-1cd64b7f1721 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:45.469451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9081a-baf1-432f-9452-1cd64b7f1721 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:45.470096 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9081a-baf1-432f-9452-1cd64b7f1721 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:45.471562 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9081a-baf1-432f-9452-1cd64b7f1721 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:45.471759 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e6aa18-a8af-4fbb-bfe1-f94169cfcc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:45.472569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6aa18-a8af-4fbb-bfe1-f94169cfcc45 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:45.473610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6aa18-a8af-4fbb-bfe1-f94169cfcc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:45.474059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6aa18-a8af-4fbb-bfe1-f94169cfcc45 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:45.475047 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e6aa18-a8af-4fbb-bfe1-f94169cfcc45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:45.475525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e6aa18-a8af-4fbb-bfe1-f94169cfcc45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:45.477152 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e6aa18-a8af-4fbb-bfe1-f94169cfcc45 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:45.491705 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 354/711] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:45.492778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e6aa18-a8af-4fbb-bfe1-f94169cfcc45 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:45.493659 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 358/712] 10.222.0.39 () {60 vars in 1132 bytes} [Mon Aug 8 02:13:45 2022] POST /identity/v3/users => generated 254 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:45.494960 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67e95f-660e-41e8-ad45-fa26cae30e88 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:45.495245 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67e95f-660e-41e8-ad45-fa26cae30e88 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:45.495517 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67e95f-660e-41e8-ad45-fa26cae30e88 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:45.501265 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 359/713] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:45.502259 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67e95f-660e-41e8-ad45-fa26cae30e88 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:45.567516 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:45.629755 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e6aa18-a8af-4fbb-bfe1-f94169cfcc45 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:45.635655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67e95f-660e-41e8-ad45-fa26cae30e88 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:45.675183 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cff3a3-3434-48b3-b77f-ae8cb85bfd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:45.675897 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cff3a3-3434-48b3-b77f-ae8cb85bfd2f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:45.676161 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cff3a3-3434-48b3-b77f-ae8cb85bfd2f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:45.676432 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cff3a3-3434-48b3-b77f-ae8cb85bfd2f None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:45.677053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cff3a3-3434-48b3-b77f-ae8cb85bfd2f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:45.677414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cff3a3-3434-48b3-b77f-ae8cb85bfd2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:45.679583 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cff3a3-3434-48b3-b77f-ae8cb85bfd2f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:45.680869 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12952d0-755c-4d7a-a75e-e2e538bf6531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:45.681473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12952d0-755c-4d7a-a75e-e2e538bf6531 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:45.681754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12952d0-755c-4d7a-a75e-e2e538bf6531 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:45.682025 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12952d0-755c-4d7a-a75e-e2e538bf6531 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:45.683102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12952d0-755c-4d7a-a75e-e2e538bf6531 None admin] RBAC: Adding query filter params (role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=c4238566a9c248ffbfa314f47f02670c) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:45.683577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12952d0-755c-4d7a-a75e-e2e538bf6531 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:45.685274 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12952d0-755c-4d7a-a75e-e2e538bf6531 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:45.686000 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 355/714] 10.222.0.39 () {56 vars in 1124 bytes} [Mon Aug 8 02:13:45 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:45.694296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cea1a5-5c42-41e2-aa1d-f85ac071e0b7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:45.694495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:45.694495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 360/715] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:45 2022] GET /identity/v3/role_assignments?user.id=c4238566a9c248ffbfa314f47f02670c&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:45.694837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cea1a5-5c42-41e2-aa1d-f85ac071e0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:45.695081 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cea1a5-5c42-41e2-aa1d-f85ac071e0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:45.720429 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-98cea1a5-5c42-41e2-aa1d-f85ac071e0b7 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:45.721701 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-98cea1a5-5c42-41e2-aa1d-f85ac071e0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:45.804311 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 356/716] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:45.828660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12952d0-755c-4d7a-a75e-e2e538bf6531 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:45.939930 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12952d0-755c-4d7a-a75e-e2e538bf6531 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:45.975376 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5078d3-b50a-4d2f-bc66-0af0cdc10f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:45.976004 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5078d3-b50a-4d2f-bc66-0af0cdc10f2d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:45.976281 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5078d3-b50a-4d2f-bc66-0af0cdc10f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:45.976555 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5078d3-b50a-4d2f-bc66-0af0cdc10f2d None admin] PATH_INFO: `/v3/services/network` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:45.977204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5078d3-b50a-4d2f-bc66-0af0cdc10f2d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:45.978654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5078d3-b50a-4d2f-bc66-0af0cdc10f2d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:45.990531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-ac5078d3-b50a-4d2f-bc66-0af0cdc10f2d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:13:45.990531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:45.990531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:13:45.990531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:13:45.990531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:13:45.990531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:13:45.990531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:13:45.990531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:13:45.990531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:13:45.990531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:45.990531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:13:45.990531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:45.990531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:45.990531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:45.990531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:45.990531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:45.990531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:45.991599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:45.991599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:45.991599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:45.991599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:45.991599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:45.991599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:45.991599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:13:45.991599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:13:45.991599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:13:45.991599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:13:45.991599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:45.991599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:45.991599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:45.991599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:45.991599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:45.991599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:45.992417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:45.992417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:45.992417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:45.992417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:45.992417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:45.992417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:45.992417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:45.992417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:45.992417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:45.992417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:45.992417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:13:45.992417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:13:45.992417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:13:45.992417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:45.992417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 361/717] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:45 2022] GET /identity/v3/services/network => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:45.995504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cea1a5-5c42-41e2-aa1d-f85ac071e0b7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:46.043794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:46.101054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cea1a5-5c42-41e2-aa1d-f85ac071e0b7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:46.137383 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d7d34-75f6-4219-97c2-f89a9af6f220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:46.137988 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d7d34-75f6-4219-97c2-f89a9af6f220 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:46.138287 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d7d34-75f6-4219-97c2-f89a9af6f220 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:46.138560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d7d34-75f6-4219-97c2-f89a9af6f220 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:46.139189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d7d34-75f6-4219-97c2-f89a9af6f220 None admin] RBAC: Adding query filter params (name=network) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:46.139491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d7d34-75f6-4219-97c2-f89a9af6f220 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:46.140862 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d7d34-75f6-4219-97c2-f89a9af6f220 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:46.145556 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 357/718] 10.222.0.39 () {56 vars in 1112 bytes} [Mon Aug 8 02:13:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:46.151155 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5078d3-b50a-4d2f-bc66-0af0cdc10f2d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:46.224854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:46.277656 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5078d3-b50a-4d2f-bc66-0af0cdc10f2d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:46.328113 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be79d5-57f3-4a96-80a4-af4e130017f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:46.328726 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be79d5-57f3-4a96-80a4-af4e130017f2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:46.329007 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be79d5-57f3-4a96-80a4-af4e130017f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:46.329277 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be79d5-57f3-4a96-80a4-af4e130017f2 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:46.329910 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be79d5-57f3-4a96-80a4-af4e130017f2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:46.330216 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be79d5-57f3-4a96-80a4-af4e130017f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:46.331621 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be79d5-57f3-4a96-80a4-af4e130017f2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:46.343311 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 362/719] 10.222.0.39 () {56 vars in 1112 bytes} [Mon Aug 8 02:13:46 2022] GET /identity/v3/services?type=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:46.932635 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59dae4-2863-4ac3-b8a9-632358deb150 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:46.933150 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59dae4-2863-4ac3-b8a9-632358deb150 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:46.933150 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59dae4-2863-4ac3-b8a9-632358deb150 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:46.935040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 358/720] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:46.940123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba0b05-d4d4-4427-8ea0-2f292b7ed950 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:46.940927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba0b05-d4d4-4427-8ea0-2f292b7ed950 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:46.941396 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba0b05-d4d4-4427-8ea0-2f292b7ed950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:47.079819 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-9fba0b05-d4d4-4427-8ea0-2f292b7ed950 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:47.081373 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9fba0b05-d4d4-4427-8ea0-2f292b7ed950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:47.155808 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 363/721] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:47.178414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59dae4-2863-4ac3-b8a9-632358deb150 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:47.323298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59dae4-2863-4ac3-b8a9-632358deb150 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:47.334642 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:47.359697 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098dba6f-825e-4a58-8d41-e767eadd1c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:47.360326 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098dba6f-825e-4a58-8d41-e767eadd1c52 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:47.360604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098dba6f-825e-4a58-8d41-e767eadd1c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:47.360874 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098dba6f-825e-4a58-8d41-e767eadd1c52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:47.371096 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098dba6f-825e-4a58-8d41-e767eadd1c52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:47.375831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-098dba6f-825e-4a58-8d41-e767eadd1c52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:47.375831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:47.375831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:47.375831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:47.375831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:47.375831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:47.375831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:47.375831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:47.375831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:47.375831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:47.375831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:47.375831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:47.375831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:47.375831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:47.375831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:47.375831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:47.376748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:47.376748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:47.376748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:47.376748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:47.376748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:47.376748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:47.376748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:47.376748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:47.376748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:47.376748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:47.376748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:47.376748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:47.376748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:47.376748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:47.376748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:47.376748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:47.377578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:47.377578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:47.377578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:47.377578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:47.377578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:47.377578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:47.377578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:47.377578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:47.377578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:47.377578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:47.377578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:47.377578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:47.377578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 359/722] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:47.385007 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba0b05-d4d4-4427-8ea0-2f292b7ed950 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:47.463446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:47.507801 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba0b05-d4d4-4427-8ea0-2f292b7ed950 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:47.544395 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e96ba0-c64d-4188-9aed-ded717d2a52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:47.545376 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e96ba0-c64d-4188-9aed-ded717d2a52a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:47.545857 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e96ba0-c64d-4188-9aed-ded717d2a52a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:47.546430 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e96ba0-c64d-4188-9aed-ded717d2a52a None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:47.547288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e96ba0-c64d-4188-9aed-ded717d2a52a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:47.547809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e96ba0-c64d-4188-9aed-ded717d2a52a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:47.549393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e96ba0-c64d-4188-9aed-ded717d2a52a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:47.562565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 364/723] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:47.564571 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098dba6f-825e-4a58-8d41-e767eadd1c52 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:47.812454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098dba6f-825e-4a58-8d41-e767eadd1c52 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:47.847605 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6a1bb-bd73-4561-857a-08c024c3e6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:47.848610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6a1bb-bd73-4561-857a-08c024c3e6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:47.849104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6a1bb-bd73-4561-857a-08c024c3e6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:47.849567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6a1bb-bd73-4561-857a-08c024c3e6d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:47.869971 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6a1bb-bd73-4561-857a-08c024c3e6d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:47.871845 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6a1bb-bd73-4561-857a-08c024c3e6d4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:47.889344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-28e6a1bb-bd73-4561-857a-08c024c3e6d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:47.889344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:47.889344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:47.889344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:47.889344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:47.889344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:47.889344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:47.889344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:47.889344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:47.889344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:47.889344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:47.889344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:47.889344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:47.889344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:47.889344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:47.889344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:47.890339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:47.890339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:47.890339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:47.890339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:47.890339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:47.890339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:47.890339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:47.890339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:47.890339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:47.890339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:47.890339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:47.890339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:47.890339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:47.890339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:47.890339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:47.890339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:47.890339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:47.891229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:47.891229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:47.891229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:47.908251 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 360/724] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:47.908935 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e96ba0-c64d-4188-9aed-ded717d2a52a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:48.038115 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e96ba0-c64d-4188-9aed-ded717d2a52a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:48.079028 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:48.103013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c027702-e2e3-4a66-8244-0083b8b30913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:48.103668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c027702-e2e3-4a66-8244-0083b8b30913 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.103949 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c027702-e2e3-4a66-8244-0083b8b30913 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.104219 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c027702-e2e3-4a66-8244-0083b8b30913 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.104870 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c027702-e2e3-4a66-8244-0083b8b30913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:48.105171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c027702-e2e3-4a66-8244-0083b8b30913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:48.106553 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c027702-e2e3-4a66-8244-0083b8b30913 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:48.118645 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 365/725] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:48.124033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6a1bb-bd73-4561-857a-08c024c3e6d4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:48.176712 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:48.277682 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6a1bb-bd73-4561-857a-08c024c3e6d4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:48.307985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118eb99-1d48-4a24-8e0f-ebbbf849d4a1 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.308287 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118eb99-1d48-4a24-8e0f-ebbbf849d4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.308531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118eb99-1d48-4a24-8e0f-ebbbf849d4a1 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.310421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 366/726] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:13:48.316322 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be193272-1cad-4dce-bc3d-02c7bfb262f2 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.316672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be193272-1cad-4dce-bc3d-02c7bfb262f2 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.317447 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be193272-1cad-4dce-bc3d-02c7bfb262f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.331025 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723e3d3-d186-401d-861a-e4b33ce22f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:48.331740 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723e3d3-d186-401d-861a-e4b33ce22f1c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.331988 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723e3d3-d186-401d-861a-e4b33ce22f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.332289 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723e3d3-d186-401d-861a-e4b33ce22f1c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.337434 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-be193272-1cad-4dce-bc3d-02c7bfb262f2 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:48.338664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-be193272-1cad-4dce-bc3d-02c7bfb262f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:48.343378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3723e3d3-d186-401d-861a-e4b33ce22f1c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:48.344830 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3723e3d3-d186-401d-861a-e4b33ce22f1c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:48.360612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3723e3d3-d186-401d-861a-e4b33ce22f1c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:13:48.360612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:48.360612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:48.360612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:48.360612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:48.360612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:48.360612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:48.360612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:48.360612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:48.360612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:48.360612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:48.360612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:48.360612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:48.360612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:48.360612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:48.360612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:48.361814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:48.361814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:48.361814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:48.361814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:48.361814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:48.361814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:48.361814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:48.361814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:48.361814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:48.361814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:48.361814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:48.361814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:48.361814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:48.361814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:48.361814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:48.361814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:48.361814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:48.361814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:48.362989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:48.362989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:48.362989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:48.362989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:48.362989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:48.362989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:48.362989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:48.362989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:48.362989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:13:48.362989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:48.365414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 361/727] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:48.366905 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e3c8f-2556-47cf-b334-bccdf0692b5f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.367325 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e3c8f-2556-47cf-b334-bccdf0692b5f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.367732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e3c8f-2556-47cf-b334-bccdf0692b5f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.369425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 362/728] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:48.374772 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf1bc5-3d86-428a-a82e-63f65c4d3e3b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.375203 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf1bc5-3d86-428a-a82e-63f65c4d3e3b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.375458 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf1bc5-3d86-428a-a82e-63f65c4d3e3b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.376926 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 363/729] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:48.377506 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf1bc5-3d86-428a-a82e-63f65c4d3e3b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:48.422965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 367/730] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:48.424626 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07159c-8662-442d-ac49-062773f34933 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.425086 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07159c-8662-442d-ac49-062773f34933 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.425531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07159c-8662-442d-ac49-062773f34933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.451628 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8f07159c-8662-442d-ac49-062773f34933 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:48.452838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8f07159c-8662-442d-ac49-062773f34933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:48.475843 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:48.498829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf1bc5-3d86-428a-a82e-63f65c4d3e3b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:48.529653 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 368/731] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:48.529653 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42766c4e-ac62-46ca-8488-744e801212bc None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.529653 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42766c4e-ac62-46ca-8488-744e801212bc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.529653 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42766c4e-ac62-46ca-8488-744e801212bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.542565 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ec84c-5257-4c8e-9cff-a19e7810df85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:48.543222 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ec84c-5257-4c8e-9cff-a19e7810df85 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.543505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ec84c-5257-4c8e-9cff-a19e7810df85 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.543819 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ec84c-5257-4c8e-9cff-a19e7810df85 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.544545 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ec84c-5257-4c8e-9cff-a19e7810df85 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:48.544907 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ec84c-5257-4c8e-9cff-a19e7810df85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:48.546553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ec84c-5257-4c8e-9cff-a19e7810df85 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:48.549074 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-42766c4e-ac62-46ca-8488-744e801212bc None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:48.551902 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-42766c4e-ac62-46ca-8488-744e801212bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:48.569331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 364/732] 10.222.0.39 () {56 vars in 1135 bytes} [Mon Aug 8 02:13:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:48.569331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ec84c-5257-4c8e-9cff-a19e7810df85 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:48.585068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:48.624612 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 369/733] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:48.625771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42766c4e-ac62-46ca-8488-744e801212bc None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:48.697025 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ec84c-5257-4c8e-9cff-a19e7810df85 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:48.733215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f717dac-4df1-4af0-bea2-f254a85b54f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:48.733830 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f717dac-4df1-4af0-bea2-f254a85b54f6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.734110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f717dac-4df1-4af0-bea2-f254a85b54f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.734409 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f717dac-4df1-4af0-bea2-f254a85b54f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.734594 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42766c4e-ac62-46ca-8488-744e801212bc None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:48.743252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:48.747075 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f717dac-4df1-4af0-bea2-f254a85b54f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:48.752395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5f717dac-4df1-4af0-bea2-f254a85b54f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:48.752395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:48.752395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:48.752395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:48.752395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:48.752395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:48.752395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:48.752395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:48.752395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:48.752395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:48.752395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:48.752395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:48.752395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:48.752395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:48.752395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:48.752395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:48.753362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:48.753362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:48.753362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:48.753362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:48.753362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:48.753362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:48.753362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:48.753362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:48.753362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:48.753362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:48.753362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:48.753362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:48.753362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:48.753362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:48.753362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:48.753362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:48.759332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:48.759332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:48.759332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:48.759332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:48.759332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:48.759332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:48.759332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:48.759332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:48.759332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:48.759332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:48.759332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:48.759332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:48.759332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 365/734] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:48.759332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f717dac-4df1-4af0-bea2-f254a85b54f6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:48.776876 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8582ec-4a9e-4ad9-9e1e-f16a3c46ea5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:48.777570 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8582ec-4a9e-4ad9-9e1e-f16a3c46ea5f None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.777928 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8582ec-4a9e-4ad9-9e1e-f16a3c46ea5f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.778285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8582ec-4a9e-4ad9-9e1e-f16a3c46ea5f None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.778938 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8582ec-4a9e-4ad9-9e1e-f16a3c46ea5f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:48.780289 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8582ec-4a9e-4ad9-9e1e-f16a3c46ea5f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:48.799558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 370/735] 10.222.0.39 () {60 vars in 1141 bytes} [Mon Aug 8 02:13:48 2022] POST /identity/v3/services => generated 251 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:48.801707 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8582ec-4a9e-4ad9-9e1e-f16a3c46ea5f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:48.848627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:48.890043 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f717dac-4df1-4af0-bea2-f254a85b54f6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:48.950127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a0d26-88b5-4127-9097-bf19321c84c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:48.950369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8582ec-4a9e-4ad9-9e1e-f16a3c46ea5f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:48.951179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a0d26-88b5-4127-9097-bf19321c84c3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.951631 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a0d26-88b5-4127-9097-bf19321c84c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.952385 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a0d26-88b5-4127-9097-bf19321c84c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.005714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3620e6-ded3-4913-95eb-3b4922682a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.006703 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3620e6-ded3-4913-95eb-3b4922682a8c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.007168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3620e6-ded3-4913-95eb-3b4922682a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.007433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3620e6-ded3-4913-95eb-3b4922682a8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.020699 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3620e6-ded3-4913-95eb-3b4922682a8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:49.027175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-af3620e6-ded3-4913-95eb-3b4922682a8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:49.027175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:49.027175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:49.027175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:49.027175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:49.027175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:49.027175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:49.027175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:49.027175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:49.027175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:49.027175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:49.027175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:49.027175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:49.027175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:49.027175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:49.027175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:49.028144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:49.028144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:49.028144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:49.028144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:49.028144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:49.028144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:49.028144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:49.028144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:49.028144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:49.028144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:49.028144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:49.028144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:49.028144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:49.028144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:49.028144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:49.028144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:49.028970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:49.028970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:49.028970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:49.028970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:49.028970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:49.028970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:49.028970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:49.028970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:49.028970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:49.028970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:49.028970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:49.028970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:49.028970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 371/736] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:49.030136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3620e6-ded3-4913-95eb-3b4922682a8c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.030666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:49.039748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a0d26-88b5-4127-9097-bf19321c84c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:49.041774 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a0d26-88b5-4127-9097-bf19321c84c3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:49.123850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:49.141427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3620e6-ded3-4913-95eb-3b4922682a8c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:49.152021 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 366/737] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:48 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:49.152867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a0d26-88b5-4127-9097-bf19321c84c3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.180510 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a9254-f2e4-4194-ae4d-68edcdd25722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.181397 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a9254-f2e4-4194-ae4d-68edcdd25722 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.181851 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a9254-f2e4-4194-ae4d-68edcdd25722 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.182319 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a9254-f2e4-4194-ae4d-68edcdd25722 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.183179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a9254-f2e4-4194-ae4d-68edcdd25722 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:49.183654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a9254-f2e4-4194-ae4d-68edcdd25722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:49.185545 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a9254-f2e4-4194-ae4d-68edcdd25722 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:49.198811 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 372/738] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:49.200395 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37679c9c-612a-46ca-a379-ead8a5d2c74d None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.201090 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37679c9c-612a-46ca-a379-ead8a5d2c74d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.201372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37679c9c-612a-46ca-a379-ead8a5d2c74d None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.202813 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 373/739] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:49.203619 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37679c9c-612a-46ca-a379-ead8a5d2c74d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.254825 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:49.262001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a0d26-88b5-4127-9097-bf19321c84c3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:49.297791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c54ac-0687-4695-bd26-e0691f2c7757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.298438 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c54ac-0687-4695-bd26-e0691f2c7757 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.298736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c54ac-0687-4695-bd26-e0691f2c7757 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.299003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c54ac-0687-4695-bd26-e0691f2c7757 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.299649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c54ac-0687-4695-bd26-e0691f2c7757 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:49.299947 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c54ac-0687-4695-bd26-e0691f2c7757 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:49.301850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c54ac-0687-4695-bd26-e0691f2c7757 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:49.310475 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 367/740] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:13:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:49.311174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c54ac-0687-4695-bd26-e0691f2c7757 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.318022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37679c9c-612a-46ca-a379-ead8a5d2c74d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:49.356194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea0499-467a-4554-afc8-bda1b50b7c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.356817 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea0499-467a-4554-afc8-bda1b50b7c8f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.357084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea0499-467a-4554-afc8-bda1b50b7c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.357353 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea0499-467a-4554-afc8-bda1b50b7c8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.368657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea0499-467a-4554-afc8-bda1b50b7c8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:49.370089 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea0499-467a-4554-afc8-bda1b50b7c8f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:49.377423 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:49.386638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-c2ea0499-467a-4554-afc8-bda1b50b7c8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:49.386638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:49.386638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:49.386638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:49.386638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:49.386638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:49.386638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:49.386638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:49.386638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:49.386638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:49.386638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:49.386638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:49.386638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:49.386638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:49.386638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:49.386638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:49.387591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:49.387591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:49.387591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:49.387591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:49.387591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:49.387591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:49.387591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:49.387591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:49.387591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:49.387591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:49.387591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:49.387591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:49.387591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:49.387591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:49.387591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:49.387591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:49.387591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:49.393289 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:49.393289 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:49.393289 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:49.393289 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 374/741] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:49.393289 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bee79c-2a3c-46aa-95a9-6c920e54b510 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.393289 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bee79c-2a3c-46aa-95a9-6c920e54b510 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.393289 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bee79c-2a3c-46aa-95a9-6c920e54b510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.408989 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-65bee79c-2a3c-46aa-95a9-6c920e54b510 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:49.410135 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-65bee79c-2a3c-46aa-95a9-6c920e54b510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:49.423099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c54ac-0687-4695-bd26-e0691f2c7757 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:49.463596 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6db28a-bee1-4d71-aeed-7beff6f2699c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.464071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6db28a-bee1-4d71-aeed-7beff6f2699c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.464345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6db28a-bee1-4d71-aeed-7beff6f2699c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.464610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6db28a-bee1-4d71-aeed-7beff6f2699c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.475231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6db28a-bee1-4d71-aeed-7beff6f2699c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:49.476666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6db28a-bee1-4d71-aeed-7beff6f2699c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:49.484933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 375/742] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:49 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:49.485769 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bee79c-2a3c-46aa-95a9-6c920e54b510 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.502555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6c6db28a-bee1-4d71-aeed-7beff6f2699c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:49.502555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:49.502555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:49.502555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:49.502555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:49.502555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:49.502555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:49.502555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:49.502555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:49.502555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:49.502555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:49.502555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:49.502555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:49.502555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:49.502555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:49.502555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:49.503530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:49.503530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:49.503530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:49.503530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:49.503530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:49.503530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:49.503530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:49.503530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:49.503530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:49.503530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:49.503530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:49.503530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:49.503530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:49.503530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:49.503530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:49.503530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:49.503530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:49.504942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:49.504942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:49.504942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:49.507134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 368/743] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:49.508083 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6db28a-bee1-4d71-aeed-7beff6f2699c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.522603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:49.597147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bee79c-2a3c-46aa-95a9-6c920e54b510 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:49.616095 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6db28a-bee1-4d71-aeed-7beff6f2699c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:49.625808 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:49.637276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350bc6a-8d79-4dc8-87b5-6a0cd1437ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.637825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350bc6a-8d79-4dc8-87b5-6a0cd1437ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.638097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350bc6a-8d79-4dc8-87b5-6a0cd1437ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.638393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350bc6a-8d79-4dc8-87b5-6a0cd1437ff6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.649969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2350bc6a-8d79-4dc8-87b5-6a0cd1437ff6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:49.650460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ceecc-f050-4b6c-bea0-3b225d59a843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.651058 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ceecc-f050-4b6c-bea0-3b225d59a843 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.651338 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ceecc-f050-4b6c-bea0-3b225d59a843 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.651436 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2350bc6a-8d79-4dc8-87b5-6a0cd1437ff6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:49.651607 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ceecc-f050-4b6c-bea0-3b225d59a843 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.652246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ceecc-f050-4b6c-bea0-3b225d59a843 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:49.652551 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ceecc-f050-4b6c-bea0-3b225d59a843 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:49.653903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ceecc-f050-4b6c-bea0-3b225d59a843 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:49.665298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 369/744] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:49.666074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ceecc-f050-4b6c-bea0-3b225d59a843 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.667069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-2350bc6a-8d79-4dc8-87b5-6a0cd1437ff6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:49.667069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:49.667069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:49.667069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:49.667069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:49.667069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:49.667069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:49.667069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:49.667069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:49.667069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:49.667069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:49.667069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:49.667069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:49.667069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:49.667069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:49.667069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:49.667994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:49.667994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:49.667994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:49.667994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:49.667994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:49.667994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:49.667994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:49.667994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:49.667994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:49.667994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:49.667994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:49.667994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:49.667994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:49.667994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:49.667994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:49.667994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:49.667994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:49.668883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:49.668883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:49.668883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:49.668883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 376/745] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:49.669364 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350bc6a-8d79-4dc8-87b5-6a0cd1437ff6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.752033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:49.780520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350bc6a-8d79-4dc8-87b5-6a0cd1437ff6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:49.785340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ceecc-f050-4b6c-bea0-3b225d59a843 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:49.815882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4553de6-1b1e-43d3-a3bb-b544eb90b6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.816473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4553de6-1b1e-43d3-a3bb-b544eb90b6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.817301 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4553de6-1b1e-43d3-a3bb-b544eb90b6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.817589 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4553de6-1b1e-43d3-a3bb-b544eb90b6d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.818222 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4553de6-1b1e-43d3-a3bb-b544eb90b6d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:49.818624 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4553de6-1b1e-43d3-a3bb-b544eb90b6d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:49.820015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4553de6-1b1e-43d3-a3bb-b544eb90b6d6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:49.821106 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92697ea5-0bf9-49db-aeca-faeea8d44781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.821704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92697ea5-0bf9-49db-aeca-faeea8d44781 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.821984 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92697ea5-0bf9-49db-aeca-faeea8d44781 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.822272 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92697ea5-0bf9-49db-aeca-faeea8d44781 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.822927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92697ea5-0bf9-49db-aeca-faeea8d44781 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:49.824266 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92697ea5-0bf9-49db-aeca-faeea8d44781 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:49.831845 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 377/746] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:49.832849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4553de6-1b1e-43d3-a3bb-b544eb90b6d6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.844615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 370/747] 10.222.0.39 () {60 vars in 1141 bytes} [Mon Aug 8 02:13:49 2022] POST /identity/v3/services => generated 240 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:49.844615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92697ea5-0bf9-49db-aeca-faeea8d44781 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.891195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:49.971553 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4553de6-1b1e-43d3-a3bb-b544eb90b6d6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:49.979438 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92697ea5-0bf9-49db-aeca-faeea8d44781 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:50.015906 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:50.022335 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2a6dc-a67c-4deb-b72c-b210c863b554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:50.024488 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8708048-3e4a-4576-8f7e-88114e7c82bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:50.024597 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2a6dc-a67c-4deb-b72c-b210c863b554 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.024597 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2a6dc-a67c-4deb-b72c-b210c863b554 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.024597 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2a6dc-a67c-4deb-b72c-b210c863b554 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.024842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8708048-3e4a-4576-8f7e-88114e7c82bb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.024842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8708048-3e4a-4576-8f7e-88114e7c82bb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.024842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8708048-3e4a-4576-8f7e-88114e7c82bb None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.024842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8708048-3e4a-4576-8f7e-88114e7c82bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:50.026291 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8708048-3e4a-4576-8f7e-88114e7c82bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:50.027636 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8708048-3e4a-4576-8f7e-88114e7c82bb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:50.032111 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2a6dc-a67c-4deb-b72c-b210c863b554 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:50.033728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2a6dc-a67c-4deb-b72c-b210c863b554 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:50.047578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 371/748] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:50.048726 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8708048-3e4a-4576-8f7e-88114e7c82bb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:50.055292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-9db2a6dc-a67c-4deb-b72c-b210c863b554 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:50.055292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:50.055292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:50.055292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:50.055292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:50.055292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:50.055292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:50.055292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:50.055292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:50.055292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:50.055292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:50.055292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:50.055292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:50.055292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:50.055292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:50.056200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:50.056200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:50.056200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:50.056200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:50.056200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:50.056200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:50.056200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:50.056200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:50.056200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:50.056200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:50.056200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:50.056200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:50.056200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:50.056200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:50.056200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:50.056200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:50.056200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:50.057060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:50.057060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:50.057060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:50.057060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:50.057060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:50.057060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:50.057060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:50.057060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:50.057060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 378/749] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:50.058758 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2a6dc-a67c-4deb-b72c-b210c863b554 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:50.183240 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8708048-3e4a-4576-8f7e-88114e7c82bb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:50.184361 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:50.187618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2a6dc-a67c-4deb-b72c-b210c863b554 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:50.223538 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca24622-0513-4829-9d2e-f0b2c0686114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:50.224310 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca24622-0513-4829-9d2e-f0b2c0686114 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.224686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca24622-0513-4829-9d2e-f0b2c0686114 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.225046 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca24622-0513-4829-9d2e-f0b2c0686114 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.232688 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca24622-0513-4829-9d2e-f0b2c0686114 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:50.234305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca24622-0513-4829-9d2e-f0b2c0686114 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:50.236007 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22941cd7-5412-4dfe-8897-e21ffcd6aed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:50.237663 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22941cd7-5412-4dfe-8897-e21ffcd6aed6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.238119 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22941cd7-5412-4dfe-8897-e21ffcd6aed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.239197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22941cd7-5412-4dfe-8897-e21ffcd6aed6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.247632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22941cd7-5412-4dfe-8897-e21ffcd6aed6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:50.247779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0ca24622-0513-4829-9d2e-f0b2c0686114 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:13:50.247779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:50.247779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:50.247779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:50.247779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:50.247779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:50.247779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:50.247779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:50.247779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:50.247779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:50.247779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:50.247779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:50.247779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:50.247779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:50.247779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:50.247779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:50.248881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:50.248881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:50.248881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:50.248881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:50.248881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:50.248881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:50.248881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:50.248881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:50.248881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:50.248881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:50.248881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:50.248881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:50.248881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:50.248881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:50.248881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:50.248881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:50.248881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:50.248881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:50.250057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:50.250057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:50.250057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:50.250057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:50.250057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:50.250057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:50.250057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:50.250057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:50.250057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:13:50.250057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:50.251565 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 372/750] 10.222.0.39 () {56 vars in 1093 bytes} [Mon Aug 8 02:13:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:50.252415 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22941cd7-5412-4dfe-8897-e21ffcd6aed6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:50.257141 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca24622-0513-4829-9d2e-f0b2c0686114 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:50.266491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-22941cd7-5412-4dfe-8897-e21ffcd6aed6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:13:50.266491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:50.266491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:50.266491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:50.266491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:50.266491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:50.266491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:50.266491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:50.266491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:50.266491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:50.266491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:50.266491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:50.266491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:50.266491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:50.266491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:50.266491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:50.267690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:50.267690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:50.267690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:50.267690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:50.267690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:50.267690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:50.267690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:50.267690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:50.267690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:50.267690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:50.267690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:50.267690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:50.267690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:50.267690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:50.267690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:50.267690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:50.267690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:50.267690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:50.268875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:50.268875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:50.268875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:50.268875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:50.268875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:50.268875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:50.268875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:50.268875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:50.268875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:13:50.268875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:50.276512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 379/751] 10.222.0.39 () {56 vars in 1117 bytes} [Mon Aug 8 02:13:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:50.276512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22941cd7-5412-4dfe-8897-e21ffcd6aed6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:50.325205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:50.394149 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca24622-0513-4829-9d2e-f0b2c0686114 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:50.398609 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22941cd7-5412-4dfe-8897-e21ffcd6aed6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:50.435835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b6cb5b-8f06-49a8-b8a9-4f65ec603252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:50.436231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c024506-416c-4358-a914-f35cc6d1ad31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:50.436833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c024506-416c-4358-a914-f35cc6d1ad31 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.437107 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c024506-416c-4358-a914-f35cc6d1ad31 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.437370 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c024506-416c-4358-a914-f35cc6d1ad31 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.437660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6cb5b-8f06-49a8-b8a9-4f65ec603252 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.438046 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c024506-416c-4358-a914-f35cc6d1ad31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:50.438382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c024506-416c-4358-a914-f35cc6d1ad31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:50.438487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6cb5b-8f06-49a8-b8a9-4f65ec603252 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.438978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6cb5b-8f06-49a8-b8a9-4f65ec603252 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.440252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c024506-416c-4358-a914-f35cc6d1ad31 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:50.440392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b6cb5b-8f06-49a8-b8a9-4f65ec603252 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:50.440895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b6cb5b-8f06-49a8-b8a9-4f65ec603252 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:50.442771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b6cb5b-8f06-49a8-b8a9-4f65ec603252 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:50.456276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 373/752] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:50.457784 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c024506-416c-4358-a914-f35cc6d1ad31 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:50.464301 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 380/753] 10.222.0.39 () {56 vars in 1133 bytes} [Mon Aug 8 02:13:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:50.465442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b6cb5b-8f06-49a8-b8a9-4f65ec603252 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:50.488158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:50.584868 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c024506-416c-4358-a914-f35cc6d1ad31 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:50.588762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b6cb5b-8f06-49a8-b8a9-4f65ec603252 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:50.593531 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:50.624893 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f787a-641c-419a-bea8-2a35a54cb38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:50.626163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f787a-641c-419a-bea8-2a35a54cb38c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.626963 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f787a-641c-419a-bea8-2a35a54cb38c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.627873 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f787a-641c-419a-bea8-2a35a54cb38c None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.629156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f787a-641c-419a-bea8-2a35a54cb38c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:50.629674 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f787a-641c-419a-bea8-2a35a54cb38c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:50.631357 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f787a-641c-419a-bea8-2a35a54cb38c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:50.632432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49146d6e-8c56-4b7b-aa8d-369eee9c1cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:50.633032 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49146d6e-8c56-4b7b-aa8d-369eee9c1cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.633306 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49146d6e-8c56-4b7b-aa8d-369eee9c1cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.633571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49146d6e-8c56-4b7b-aa8d-369eee9c1cf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.634599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49146d6e-8c56-4b7b-aa8d-369eee9c1cf7 None admin] RBAC: Adding query filter params (role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=ac60b4d621234225830c94edfac400e2) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:50.634964 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49146d6e-8c56-4b7b-aa8d-369eee9c1cf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:50.637295 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49146d6e-8c56-4b7b-aa8d-369eee9c1cf7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:50.648916 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 381/754] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:50 2022] GET /identity/v3/role_assignments?user.id=ac60b4d621234225830c94edfac400e2&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:50.648916 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49146d6e-8c56-4b7b-aa8d-369eee9c1cf7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:50.663146 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 374/755] 10.222.0.39 () {56 vars in 1149 bytes} [Mon Aug 8 02:13:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:50.670649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f787a-641c-419a-bea8-2a35a54cb38c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:50.788250 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:50.810700 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49146d6e-8c56-4b7b-aa8d-369eee9c1cf7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:50.854541 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070797a2-c012-44d9-805e-b62759a56fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:50.855425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070797a2-c012-44d9-805e-b62759a56fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.855922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070797a2-c012-44d9-805e-b62759a56fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.856418 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070797a2-c012-44d9-805e-b62759a56fb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.862270 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f787a-641c-419a-bea8-2a35a54cb38c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:50.868854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070797a2-c012-44d9-805e-b62759a56fb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:50.870604 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070797a2-c012-44d9-805e-b62759a56fb3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:50.885861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-070797a2-c012-44d9-805e-b62759a56fb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:50.885861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:50.885861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:50.885861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:50.885861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:50.885861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:50.885861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:50.885861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:50.885861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:50.885861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:50.885861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:50.885861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:50.885861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:50.885861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:50.885861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:50.885861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:50.886849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:50.886849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:50.886849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:50.886849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:50.886849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:50.886849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:50.886849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:50.886849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:50.886849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:50.886849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:50.886849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:50.886849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:50.886849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:50.886849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:50.886849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:50.886849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:50.886849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:50.887720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:50.887720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:50.887720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:50.889762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 382/756] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:50.898615 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070797a2-c012-44d9-805e-b62759a56fb3 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:50.900827 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c41371-5acd-4c3d-bf15-c8e2e8e24d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:50.901505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c41371-5acd-4c3d-bf15-c8e2e8e24d72 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.901833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c41371-5acd-4c3d-bf15-c8e2e8e24d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.902111 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c41371-5acd-4c3d-bf15-c8e2e8e24d72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.937805 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:50.977511 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c41371-5acd-4c3d-bf15-c8e2e8e24d72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:50.979199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c41371-5acd-4c3d-bf15-c8e2e8e24d72 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:51.015955 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070797a2-c012-44d9-805e-b62759a56fb3 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:51.050750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9b80f-59bb-4224-832d-8f242b6802fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:51.051312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9b80f-59bb-4224-832d-8f242b6802fc None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:51.051591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9b80f-59bb-4224-832d-8f242b6802fc None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:51.051866 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9b80f-59bb-4224-832d-8f242b6802fc None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:51.052515 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9b80f-59bb-4224-832d-8f242b6802fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:51.052820 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9b80f-59bb-4224-832d-8f242b6802fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:51.054175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9b80f-59bb-4224-832d-8f242b6802fc None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:51.063958 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:51.068985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 383/757] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:51.070851 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94587a39-1380-43fb-a774-a4a330334cf1 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:51.071359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94587a39-1380-43fb-a774-a4a330334cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:51.071740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94587a39-1380-43fb-a774-a4a330334cf1 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:51.073524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 384/758] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:51.074891 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94587a39-1380-43fb-a774-a4a330334cf1 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:51.105722 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 375/759] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:50 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:13:51.108276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb4e01-abc0-4c6d-8906-0c916ba93f1d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:51.108557 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb4e01-abc0-4c6d-8906-0c916ba93f1d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:51.108831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb4e01-abc0-4c6d-8906-0c916ba93f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:51.131039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f3fb4e01-abc0-4c6d-8906-0c916ba93f1d None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:51.132236 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f3fb4e01-abc0-4c6d-8906-0c916ba93f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:51.199466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:51.210366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 376/760] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:13:51.211407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb4e01-abc0-4c6d-8906-0c916ba93f1d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:51.213004 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94587a39-1380-43fb-a774-a4a330334cf1 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:51.259922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ad36f-c7ca-46ff-9802-8ccb1745682f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:51.260887 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ad36f-c7ca-46ff-9802-8ccb1745682f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:51.261311 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ad36f-c7ca-46ff-9802-8ccb1745682f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:51.261719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ad36f-c7ca-46ff-9802-8ccb1745682f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:51.270068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ad36f-c7ca-46ff-9802-8ccb1745682f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:51.272343 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ad36f-c7ca-46ff-9802-8ccb1745682f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:51.286978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-6c4ad36f-c7ca-46ff-9802-8ccb1745682f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:51.286978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:51.286978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:51.286978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:51.286978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:51.286978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:51.286978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:51.286978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:51.286978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:51.286978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:51.286978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:51.286978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:51.286978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:51.286978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:51.286978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:51.287871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:51.287871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:51.287871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:51.287871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:51.287871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:51.287871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:51.287871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:51.287871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:51.287871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:51.287871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:51.287871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:51.287871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:51.287871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:51.287871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:51.287871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:51.287871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:51.287871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:51.288755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:51.288755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:51.288755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:51.288755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:51.288755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:51.288755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:51.288755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:51.288755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:51.288755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 385/761] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:51.289670 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ad36f-c7ca-46ff-9802-8ccb1745682f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:51.325788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb4e01-abc0-4c6d-8906-0c916ba93f1d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:51.360527 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:51.365087 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e2d3d-2dce-43d0-bf0b-c2359deee131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:51.365713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e2d3d-2dce-43d0-bf0b-c2359deee131 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:51.366053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e2d3d-2dce-43d0-bf0b-c2359deee131 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:51.366349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e2d3d-2dce-43d0-bf0b-c2359deee131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:51.383069 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832e2d3d-2dce-43d0-bf0b-c2359deee131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:51.384983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832e2d3d-2dce-43d0-bf0b-c2359deee131 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:51.410758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-832e2d3d-2dce-43d0-bf0b-c2359deee131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:51.410758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:51.410758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:51.410758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:51.410758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:51.410758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:51.410758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:51.410758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:51.410758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:51.410758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:51.410758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:51.410758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:51.410758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:51.410758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:51.410758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:51.410758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:51.412959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:51.412959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:51.412959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:51.412959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:51.412959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:51.412959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:51.412959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:51.412959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:51.412959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:51.412959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:51.412959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:51.412959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:51.412959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:51.412959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:51.412959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:51.412959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:51.412959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:51.415143 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:51.415143 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:51.415143 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:51.415523 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ad36f-c7ca-46ff-9802-8ccb1745682f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:51.415670 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 377/762] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:51.415670 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e2d3d-2dce-43d0-bf0b-c2359deee131 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:51.469065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40db38-ba8a-436c-b48d-caa2ad4783d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:51.469992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40db38-ba8a-436c-b48d-caa2ad4783d6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:51.470441 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40db38-ba8a-436c-b48d-caa2ad4783d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:51.470963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40db38-ba8a-436c-b48d-caa2ad4783d6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:51.471736 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40db38-ba8a-436c-b48d-caa2ad4783d6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:51.473326 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40db38-ba8a-436c-b48d-caa2ad4783d6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:51.477935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:51.483302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-fc40db38-ba8a-436c-b48d-caa2ad4783d6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:13:51.483302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:51.483302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:13:51.483302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:13:51.483302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:13:51.483302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:13:51.483302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:13:51.483302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:13:51.483302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:13:51.483302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:51.483302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:13:51.483302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:51.483302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:51.483302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:51.483302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:51.483302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:51.483302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:51.484270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:51.484270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:51.484270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:51.484270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:51.484270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:51.484270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:51.484270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:13:51.484270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:13:51.484270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:13:51.484270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:13:51.484270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:51.484270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:51.484270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:51.484270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:51.484270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:51.484270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:51.485077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:51.485077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:51.485077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:51.485077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:51.485077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:51.485077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:51.485077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:51.485077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:51.485077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:51.485077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:51.485077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:13:51.485077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:13:51.485077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:13:51.485077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:51.487689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 386/763] 10.222.0.39 () {56 vars in 1129 bytes} [Mon Aug 8 02:13:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:51.489250 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40db38-ba8a-436c-b48d-caa2ad4783d6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:51.537159 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e2d3d-2dce-43d0-bf0b-c2359deee131 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:51.574171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b7a1f-a302-4ddb-87b4-1b4271a23f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:51.574883 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b7a1f-a302-4ddb-87b4-1b4271a23f13 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:51.575168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b7a1f-a302-4ddb-87b4-1b4271a23f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:51.575541 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b7a1f-a302-4ddb-87b4-1b4271a23f13 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:51.576288 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b7a1f-a302-4ddb-87b4-1b4271a23f13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:51.576540 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b7a1f-a302-4ddb-87b4-1b4271a23f13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:51.578054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b7a1f-a302-4ddb-87b4-1b4271a23f13 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:51.595402 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 378/764] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:51.596239 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b7a1f-a302-4ddb-87b4-1b4271a23f13 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:51.597134 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40db38-ba8a-436c-b48d-caa2ad4783d6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:51.616894 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:51.635100 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fabfb-5fa6-41e6-9c58-8a4b0a920434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:51.635983 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fabfb-5fa6-41e6-9c58-8a4b0a920434 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:51.636434 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fabfb-5fa6-41e6-9c58-8a4b0a920434 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:51.636917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fabfb-5fa6-41e6-9c58-8a4b0a920434 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:51.637772 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131fabfb-5fa6-41e6-9c58-8a4b0a920434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:51.638368 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131fabfb-5fa6-41e6-9c58-8a4b0a920434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:51.639971 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131fabfb-5fa6-41e6-9c58-8a4b0a920434 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:51.652675 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 387/765] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:51.653729 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fabfb-5fa6-41e6-9c58-8a4b0a920434 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:51.708933 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b7a1f-a302-4ddb-87b4-1b4271a23f13 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:51.716511 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:51.747327 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f58c2-4705-479e-9486-356b1d60e5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:51.748072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f58c2-4705-479e-9486-356b1d60e5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:51.748351 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f58c2-4705-479e-9486-356b1d60e5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:51.748674 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f58c2-4705-479e-9486-356b1d60e5d2 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:51.749303 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f58c2-4705-479e-9486-356b1d60e5d2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:51.749602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f58c2-4705-479e-9486-356b1d60e5d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:51.751033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f58c2-4705-479e-9486-356b1d60e5d2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:51.756152 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 379/766] 10.222.0.39 () {56 vars in 1124 bytes} [Mon Aug 8 02:13:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:51.758032 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f58c2-4705-479e-9486-356b1d60e5d2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:51.767664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fabfb-5fa6-41e6-9c58-8a4b0a920434 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:51.804052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedc517-b512-4108-b88a-c6b722b7e94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:51.805002 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedc517-b512-4108-b88a-c6b722b7e94e None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:51.805458 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedc517-b512-4108-b88a-c6b722b7e94e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:51.805904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedc517-b512-4108-b88a-c6b722b7e94e None admin] PATH_INFO: `/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/c4238566a9c248ffbfa314f47f02670c/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:51.808021 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedc517-b512-4108-b88a-c6b722b7e94e None admin] RBAC: Authorizing `identity:create_grant(project_id=7d474b61f1fe4f17aacb7a3665315c79, user_id=c4238566a9c248ffbfa314f47f02670c, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:51.809885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedc517-b512-4108-b88a-c6b722b7e94e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:51.834364 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 388/767] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:13:51 2022] PUT /identity/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/c4238566a9c248ffbfa314f47f02670c/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:51.834364 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedc517-b512-4108-b88a-c6b722b7e94e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:51.912150 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:51.936847 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f58c2-4705-479e-9486-356b1d60e5d2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:52.005560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedc517-b512-4108-b88a-c6b722b7e94e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:52.029237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b194a-0433-4694-91e9-b1a0a754d492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:52.029883 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b194a-0433-4694-91e9-b1a0a754d492 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.030163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b194a-0433-4694-91e9-b1a0a754d492 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.030463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b194a-0433-4694-91e9-b1a0a754d492 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.038678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b194a-0433-4694-91e9-b1a0a754d492 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:52.041030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b194a-0433-4694-91e9-b1a0a754d492 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:52.046130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db80f4-2956-46d7-b496-b7b999600d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:52.047052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db80f4-2956-46d7-b496-b7b999600d89 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.047547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db80f4-2956-46d7-b496-b7b999600d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.048010 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db80f4-2956-46d7-b496-b7b999600d89 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.048932 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db80f4-2956-46d7-b496-b7b999600d89 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:52.049440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db80f4-2956-46d7-b496-b7b999600d89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:52.051052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db80f4-2956-46d7-b496-b7b999600d89 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:52.056492 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 389/768] 10.222.0.39 () {56 vars in 1124 bytes} [Mon Aug 8 02:13:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:52.062082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ed0b194a-0433-4694-91e9-b1a0a754d492 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:52.062082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:52.062082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:52.062082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:52.062082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:52.062082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:52.062082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:52.062082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:52.062082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:52.062082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:52.062082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:52.062082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:52.062082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:52.062082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:52.062082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:52.062976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:52.062976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:52.062976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:52.062976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:52.062976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:52.062976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:52.062976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:52.062976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:52.062976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:52.062976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:52.062976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:52.062976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:52.062976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:52.062976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:52.062976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:52.062976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:52.062976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:52.063853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:52.063853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:52.063853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:52.063853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:52.063853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:52.063853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:52.063853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:52.063853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:52.063853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 380/769] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:52.065654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558099c-8acf-49f2-bb76-a099b3c3e463 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.066177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558099c-8acf-49f2-bb76-a099b3c3e463 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.066704 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558099c-8acf-49f2-bb76-a099b3c3e463 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.068440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 390/770] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:52.069295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b194a-0433-4694-91e9-b1a0a754d492 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:52.070799 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558099c-8acf-49f2-bb76-a099b3c3e463 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:52.110756 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:52.177740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558099c-8acf-49f2-bb76-a099b3c3e463 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:52.179757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:52.181469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b194a-0433-4694-91e9-b1a0a754d492 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:52.215449 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb17328-58d2-45a8-950e-ad6f0911c699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:52.216824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb17328-58d2-45a8-950e-ad6f0911c699 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.217362 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb17328-58d2-45a8-950e-ad6f0911c699 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.217955 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb17328-58d2-45a8-950e-ad6f0911c699 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.218918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb17328-58d2-45a8-950e-ad6f0911c699 None admin] RBAC: Adding query filter params (interface=public, service_id=b62ddcb86f2547c78ae58c6d3638bd23, region_id=RegionOne) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:52.219441 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb17328-58d2-45a8-950e-ad6f0911c699 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:52.219853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d286f8-17c4-46f1-a6cc-4b324caf5ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:52.220568 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d286f8-17c4-46f1-a6cc-4b324caf5ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.220946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d286f8-17c4-46f1-a6cc-4b324caf5ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.221313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d286f8-17c4-46f1-a6cc-4b324caf5ec0 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.221903 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb17328-58d2-45a8-950e-ad6f0911c699 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:52.222277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d286f8-17c4-46f1-a6cc-4b324caf5ec0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:52.223293 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d286f8-17c4-46f1-a6cc-4b324caf5ec0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:52.225026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d286f8-17c4-46f1-a6cc-4b324caf5ec0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:52.231165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 391/771] 10.222.0.39 () {56 vars in 1251 bytes} [Mon Aug 8 02:13:52 2022] GET /identity/v3/endpoints?service_id=b62ddcb86f2547c78ae58c6d3638bd23&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:52.232566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9de364-222e-4718-877c-8fb58c096d4c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.232997 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9de364-222e-4718-877c-8fb58c096d4c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.233413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9de364-222e-4718-877c-8fb58c096d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.240475 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 381/772] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:52.246197 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d286f8-17c4-46f1-a6cc-4b324caf5ec0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:52.253036 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7d9de364-222e-4718-877c-8fb58c096d4c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:52.254515 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7d9de364-222e-4718-877c-8fb58c096d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:52.324756 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 392/773] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:13:52.325885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9de364-222e-4718-877c-8fb58c096d4c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:52.346718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:52.359319 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d286f8-17c4-46f1-a6cc-4b324caf5ec0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:52.392161 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c0994-8805-49d6-a3b4-b59794da7860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:52.392821 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c0994-8805-49d6-a3b4-b59794da7860 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.393199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c0994-8805-49d6-a3b4-b59794da7860 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.393569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c0994-8805-49d6-a3b4-b59794da7860 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.395237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c0994-8805-49d6-a3b4-b59794da7860 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:52.396463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c0994-8805-49d6-a3b4-b59794da7860 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:52.402626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 382/774] 10.222.0.39 () {56 vars in 1087 bytes} [Mon Aug 8 02:13:52 2022] GET /identity/v3/services => generated 772 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:52.403865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c0994-8805-49d6-a3b4-b59794da7860 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:52.461151 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:52.471253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9de364-222e-4718-877c-8fb58c096d4c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:52.518434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c0994-8805-49d6-a3b4-b59794da7860 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:52.520579 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb68c8a-61e5-43a6-bfb6-b211b753da56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:52.521480 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb68c8a-61e5-43a6-bfb6-b211b753da56 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.530290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb68c8a-61e5-43a6-bfb6-b211b753da56 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.530290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb68c8a-61e5-43a6-bfb6-b211b753da56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.530290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb68c8a-61e5-43a6-bfb6-b211b753da56 None admin] RBAC: Adding query filter params (role.id=838b081e674f4c0ab4a33bcf8e614dc5, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=b88ccd3e5c4c48d3a27f4eaa7874e28e) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:52.530290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb68c8a-61e5-43a6-bfb6-b211b753da56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:52.530290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb68c8a-61e5-43a6-bfb6-b211b753da56 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:52.532377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 393/775] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:52 2022] GET /identity/v3/role_assignments?user.id=b88ccd3e5c4c48d3a27f4eaa7874e28e&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=838b081e674f4c0ab4a33bcf8e614dc5 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:52.589160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b604858-8f8a-4252-b971-2909378d3f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:52.589813 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b604858-8f8a-4252-b971-2909378d3f1f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.590094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b604858-8f8a-4252-b971-2909378d3f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.590393 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b604858-8f8a-4252-b971-2909378d3f1f None admin] PATH_INFO: `/v3/services/network` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.590992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b604858-8f8a-4252-b971-2909378d3f1f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:52.592382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b604858-8f8a-4252-b971-2909378d3f1f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:52.609702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6b604858-8f8a-4252-b971-2909378d3f1f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:13:52.609702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:52.609702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:13:52.609702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:13:52.609702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:13:52.609702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:13:52.609702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:13:52.609702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:13:52.609702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:13:52.609702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:52.609702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:13:52.609702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:52.609702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:52.609702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:52.609702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:52.609702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:52.609702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:52.610767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:52.610767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:52.610767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:52.610767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:52.610767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:52.610767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:52.610767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:13:52.610767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:13:52.610767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:13:52.610767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:13:52.610767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:52.610767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:52.610767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:52.610767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:52.610767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:52.610767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:52.611622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:52.611622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:52.611622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:52.611622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:52.611622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:52.611622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:52.611622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:52.611622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:52.611622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:52.611622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:52.611622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:13:52.611622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:13:52.611622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:13:52.611622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:52.611622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 383/776] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:52 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:52.619493 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb68c8a-61e5-43a6-bfb6-b211b753da56 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:52.742507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb68c8a-61e5-43a6-bfb6-b211b753da56 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:52.779824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8924eb-6f98-4cf7-9742-3502340188d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:52.780430 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8924eb-6f98-4cf7-9742-3502340188d9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.780711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8924eb-6f98-4cf7-9742-3502340188d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.780985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8924eb-6f98-4cf7-9742-3502340188d9 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.781625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8924eb-6f98-4cf7-9742-3502340188d9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:52.781932 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8924eb-6f98-4cf7-9742-3502340188d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:52.783338 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8924eb-6f98-4cf7-9742-3502340188d9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:52.788713 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 394/777] 10.222.0.39 () {56 vars in 1112 bytes} [Mon Aug 8 02:13:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:52.794223 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b604858-8f8a-4252-b971-2909378d3f1f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:52.815128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:52.900339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b604858-8f8a-4252-b971-2909378d3f1f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:52.935514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be979b3-6b4d-438a-b5de-22c2cbe4749c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:52.936121 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be979b3-6b4d-438a-b5de-22c2cbe4749c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.936400 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be979b3-6b4d-438a-b5de-22c2cbe4749c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.936673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be979b3-6b4d-438a-b5de-22c2cbe4749c None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.937304 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be979b3-6b4d-438a-b5de-22c2cbe4749c None admin] RBAC: Adding query filter params (type=network) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:52.937609 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be979b3-6b4d-438a-b5de-22c2cbe4749c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:52.939016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be979b3-6b4d-438a-b5de-22c2cbe4749c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:52.945041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 384/778] 10.222.0.39 () {56 vars in 1112 bytes} [Mon Aug 8 02:13:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:52.950560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8924eb-6f98-4cf7-9742-3502340188d9 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:52.955518 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e1c24-22f2-4696-85a9-04dade865155 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.955811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e1c24-22f2-4696-85a9-04dade865155 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.956087 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e1c24-22f2-4696-85a9-04dade865155 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.958269 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 385/779] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:52.962668 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e14633-5a91-45ed-81f8-ff961e8bdaad None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.962953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e14633-5a91-45ed-81f8-ff961e8bdaad None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.963233 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e14633-5a91-45ed-81f8-ff961e8bdaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.981223 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-99e14633-5a91-45ed-81f8-ff961e8bdaad None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:52.982463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-99e14633-5a91-45ed-81f8-ff961e8bdaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:53.006277 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:53.121492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 386/780] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:13:53.133229 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8924eb-6f98-4cf7-9742-3502340188d9 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:53.148867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e14633-5a91-45ed-81f8-ff961e8bdaad None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:53.168191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:53.201069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c143f-07c0-448a-a728-3c12064eb3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:53.201719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c143f-07c0-448a-a728-3c12064eb3cb None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:53.202000 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c143f-07c0-448a-a728-3c12064eb3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:53.202288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c143f-07c0-448a-a728-3c12064eb3cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:53.202988 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c143f-07c0-448a-a728-3c12064eb3cb None admin] RBAC: Adding query filter params (interface=public, service_id=0a121f381b3f4e3d88d17dfc43e76ed5, region_id=RegionOne) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:53.203293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c143f-07c0-448a-a728-3c12064eb3cb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:53.204814 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c143f-07c0-448a-a728-3c12064eb3cb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:53.215290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 395/781] 10.222.0.39 () {56 vars in 1251 bytes} [Mon Aug 8 02:13:52 2022] GET /identity/v3/endpoints?service_id=0a121f381b3f4e3d88d17dfc43e76ed5&interface=public®ion_id=RegionOne => generated 194 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:53.220939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c143f-07c0-448a-a728-3c12064eb3cb None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:53.266117 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e14633-5a91-45ed-81f8-ff961e8bdaad None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:53.305297 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa39c6c-0535-46ba-9f50-cbd804a267fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:53.305922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa39c6c-0535-46ba-9f50-cbd804a267fe None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:53.306201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa39c6c-0535-46ba-9f50-cbd804a267fe None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:53.306498 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa39c6c-0535-46ba-9f50-cbd804a267fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:53.325039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa39c6c-0535-46ba-9f50-cbd804a267fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:53.330495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-aaa39c6c-0535-46ba-9f50-cbd804a267fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:53.330495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:53.330495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:53.330495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:53.330495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:53.330495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:53.330495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:53.330495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:53.330495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:53.330495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:53.330495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:53.330495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:53.330495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:53.330495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:53.330495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:53.330495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:53.331405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:53.331405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:53.331405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:53.331405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:53.331405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:53.331405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:53.331405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:53.331405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:53.331405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:53.331405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:53.331405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:53.331405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:53.331405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:53.331405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:53.331405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:53.331405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:53.332492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:53.332492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:53.332492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:53.332492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:53.332492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:53.332492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:53.332492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:53.332492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:53.332492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:53.332492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:53.332492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:53.332492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:53.332492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 387/782] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:53.339979 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa39c6c-0535-46ba-9f50-cbd804a267fe None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:53.389252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:53.428038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:53.430189 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c143f-07c0-448a-a728-3c12064eb3cb None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:53.461162 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa39c6c-0535-46ba-9f50-cbd804a267fe None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:53.497418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808202d-63ea-45d6-90ad-aa03e52cbaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:53.498016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808202d-63ea-45d6-90ad-aa03e52cbaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:53.498320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808202d-63ea-45d6-90ad-aa03e52cbaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:53.498596 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808202d-63ea-45d6-90ad-aa03e52cbaa0 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:53.499256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808202d-63ea-45d6-90ad-aa03e52cbaa0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:53.499624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808202d-63ea-45d6-90ad-aa03e52cbaa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:53.501104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808202d-63ea-45d6-90ad-aa03e52cbaa0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:53.510187 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 388/783] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:53.515785 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808202d-63ea-45d6-90ad-aa03e52cbaa0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:53.527638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b283d-e85b-44a8-8e99-e928c4148f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:53.528185 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b283d-e85b-44a8-8e99-e928c4148f33 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:53.528456 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b283d-e85b-44a8-8e99-e928c4148f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:53.528722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b283d-e85b-44a8-8e99-e928c4148f33 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:53.529325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b283d-e85b-44a8-8e99-e928c4148f33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:53.530710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b283d-e85b-44a8-8e99-e928c4148f33 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:53.549064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 396/784] 10.222.0.39 () {56 vars in 1087 bytes} [Mon Aug 8 02:13:53 2022] GET /identity/v3/services => generated 772 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:53.625438 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808202d-63ea-45d6-90ad-aa03e52cbaa0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:53.664423 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e131b0-7844-4466-a647-3f71a564b5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:53.665080 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e131b0-7844-4466-a647-3f71a564b5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:53.665351 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e131b0-7844-4466-a647-3f71a564b5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:53.665625 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e131b0-7844-4466-a647-3f71a564b5a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:53.667858 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:53.678049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e131b0-7844-4466-a647-3f71a564b5a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:53.679667 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e131b0-7844-4466-a647-3f71a564b5a6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:53.695884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f3e131b0-7844-4466-a647-3f71a564b5a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:53.695884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:53.695884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:53.695884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:53.695884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:53.695884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:53.695884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:53.695884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:53.695884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:53.695884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:53.695884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:53.695884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:53.695884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:53.695884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:53.695884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:53.695884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:53.696903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:53.696903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:53.696903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:53.696903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:53.696903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:53.696903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:53.696903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:53.696903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:53.696903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:53.696903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:53.696903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:53.696903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:53.696903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:53.696903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:53.696903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:53.696903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:53.696903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:53.697765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:53.697765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:53.697765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:53.697765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 389/785] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:53.707973 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b283d-e85b-44a8-8e99-e928c4148f33 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:53.854446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b283d-e85b-44a8-8e99-e928c4148f33 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:53.904070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6442b7-55fa-42ab-9d4e-acf5b38e0a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:53.904694 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6442b7-55fa-42ab-9d4e-acf5b38e0a8a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:53.904972 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6442b7-55fa-42ab-9d4e-acf5b38e0a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:53.905244 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6442b7-55fa-42ab-9d4e-acf5b38e0a8a None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:53.905907 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6442b7-55fa-42ab-9d4e-acf5b38e0a8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:53.906209 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6442b7-55fa-42ab-9d4e-acf5b38e0a8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:53.907618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6442b7-55fa-42ab-9d4e-acf5b38e0a8a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:53.929613 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 397/786] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:53.935293 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e131b0-7844-4466-a647-3f71a564b5a6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:54.100009 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e131b0-7844-4466-a647-3f71a564b5a6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:54.153941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3f255-9e5e-48be-843c-32207da12a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:54.154518 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3f255-9e5e-48be-843c-32207da12a1c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:54.154799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3f255-9e5e-48be-843c-32207da12a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:54.155072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3f255-9e5e-48be-843c-32207da12a1c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:54.166075 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3f255-9e5e-48be-843c-32207da12a1c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:54.167573 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3f255-9e5e-48be-843c-32207da12a1c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:54.182004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-cac3f255-9e5e-48be-843c-32207da12a1c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:13:54.182004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:54.182004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:54.182004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:54.182004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:54.182004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:54.182004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:54.182004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:54.182004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:54.182004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:54.182004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:54.182004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:54.182004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:54.182004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:54.182004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:54.182004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:54.182990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:54.182990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:54.182990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:54.182990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:54.182990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:54.182990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:54.182990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:54.182990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:54.182990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:54.182990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:54.182990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:54.182990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:54.182990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:54.182990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:54.182990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:54.182990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:54.182990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:54.182990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:54.183912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:54.183912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:54.183912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:54.183912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:54.183912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:54.183912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:54.183912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:54.183912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:54.183912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:13:54.183912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:54.190637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 390/787] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:54.192190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6442b7-55fa-42ab-9d4e-acf5b38e0a8a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:54.218645 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:54.224231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871601f6-f892-4649-878b-702e2b9410b4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:54.224552 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871601f6-f892-4649-878b-702e2b9410b4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:54.224848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871601f6-f892-4649-878b-702e2b9410b4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:54.226400 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 391/788] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:54.231285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd80fcc-3d3a-4d40-8c95-67501947446d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:54.231565 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd80fcc-3d3a-4d40-8c95-67501947446d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:54.231845 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd80fcc-3d3a-4d40-8c95-67501947446d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:54.266086 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fdd80fcc-3d3a-4d40-8c95-67501947446d None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:54.267344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fdd80fcc-3d3a-4d40-8c95-67501947446d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:54.345566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:54.376202 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6442b7-55fa-42ab-9d4e-acf5b38e0a8a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:54.417033 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873f443-e0b0-4ffe-8d5e-2ef85f0a6a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:54.417033 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873f443-e0b0-4ffe-8d5e-2ef85f0a6a8c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:54.417033 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873f443-e0b0-4ffe-8d5e-2ef85f0a6a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:54.417033 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873f443-e0b0-4ffe-8d5e-2ef85f0a6a8c None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:54.417562 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873f443-e0b0-4ffe-8d5e-2ef85f0a6a8c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:54.417736 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873f443-e0b0-4ffe-8d5e-2ef85f0a6a8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:54.419202 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873f443-e0b0-4ffe-8d5e-2ef85f0a6a8c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:54.427996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 392/789] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:13:54.445585 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 398/790] 10.222.0.39 () {56 vars in 1135 bytes} [Mon Aug 8 02:13:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:54.450741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd80fcc-3d3a-4d40-8c95-67501947446d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:54.455973 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873f443-e0b0-4ffe-8d5e-2ef85f0a6a8c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:54.539536 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:54.598593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873f443-e0b0-4ffe-8d5e-2ef85f0a6a8c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:54.668806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408b726-33a3-4b05-b383-5b524579a26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:54.669346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408b726-33a3-4b05-b383-5b524579a26f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:54.669622 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408b726-33a3-4b05-b383-5b524579a26f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:54.669893 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408b726-33a3-4b05-b383-5b524579a26f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:54.673493 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd80fcc-3d3a-4d40-8c95-67501947446d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:54.693906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408b726-33a3-4b05-b383-5b524579a26f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:54.700812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-3408b726-33a3-4b05-b383-5b524579a26f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:54.700812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:54.700812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:54.700812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:54.700812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:54.700812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:54.700812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:54.700812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:54.700812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:54.700812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:54.700812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:54.700812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:54.700812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:54.700812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:54.700812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:54.700812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:54.701761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:54.701761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:54.701761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:54.701761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:54.701761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:54.701761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:54.701761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:54.701761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:54.701761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:54.701761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:54.701761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:54.701761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:54.701761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:54.701761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:54.701761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:54.701761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:54.702623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:54.702623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:54.702623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:54.702623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:54.702623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:54.702623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:54.702623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:54.702623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:54.702623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:54.702623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:54.702623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:54.702623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:54.702623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 399/791] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:54.708293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408b726-33a3-4b05-b383-5b524579a26f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:54.741450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf8ba9-9535-44f1-83e9-be6954c85fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:54.743273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf8ba9-9535-44f1-83e9-be6954c85fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:54.744532 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf8ba9-9535-44f1-83e9-be6954c85fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:54.746368 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf8ba9-9535-44f1-83e9-be6954c85fd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:54.763905 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf8ba9-9535-44f1-83e9-be6954c85fd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:54.765767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf8ba9-9535-44f1-83e9-be6954c85fd1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:54.793946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6acf8ba9-9535-44f1-83e9-be6954c85fd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:54.793946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:54.793946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:54.793946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:54.793946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:54.793946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:54.793946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:54.793946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:54.793946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:54.793946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:54.793946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:54.793946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:54.793946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:54.793946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:54.793946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:54.793946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:54.794968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:54.794968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:54.794968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:54.794968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:54.794968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:54.794968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:54.794968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:54.794968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:54.794968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:54.794968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:54.794968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:54.794968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:54.794968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:54.794968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:54.794968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:54.794968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:54.794968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:54.795836 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:54.795836 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:54.795836 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:54.799932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 393/792] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:54.804342 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf8ba9-9535-44f1-83e9-be6954c85fd1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:54.839293 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:54.921568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408b726-33a3-4b05-b383-5b524579a26f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:54.945708 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf8ba9-9535-44f1-83e9-be6954c85fd1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:54.976163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:54.983413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccc34a-e3c1-44bd-8a4a-a0ed440b35e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:54.984115 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccc34a-e3c1-44bd-8a4a-a0ed440b35e1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:54.984491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccc34a-e3c1-44bd-8a4a-a0ed440b35e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:54.984862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccc34a-e3c1-44bd-8a4a-a0ed440b35e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:54.985623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccc34a-e3c1-44bd-8a4a-a0ed440b35e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:54.986022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccc34a-e3c1-44bd-8a4a-a0ed440b35e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:54.987510 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccc34a-e3c1-44bd-8a4a-a0ed440b35e1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:54.992969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d615d-bd98-4ad8-93bb-d58ff4ef2d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:54.993579 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d615d-bd98-4ad8-93bb-d58ff4ef2d41 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:54.993862 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d615d-bd98-4ad8-93bb-d58ff4ef2d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:54.994731 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d615d-bd98-4ad8-93bb-d58ff4ef2d41 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:54.995393 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d615d-bd98-4ad8-93bb-d58ff4ef2d41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:54.995701 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d615d-bd98-4ad8-93bb-d58ff4ef2d41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:54.997355 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d615d-bd98-4ad8-93bb-d58ff4ef2d41 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:54.998167 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 400/793] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:55.005081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e509436-606b-4bd1-8342-64aa500e6b92 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.005522 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e509436-606b-4bd1-8342-64aa500e6b92 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.005893 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e509436-606b-4bd1-8342-64aa500e6b92 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.007792 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 401/794] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:55.008619 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e509436-606b-4bd1-8342-64aa500e6b92 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:55.026376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 394/795] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:55.027806 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420ecf5-525c-42a0-91c4-3c816f2b8f2d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.028088 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420ecf5-525c-42a0-91c4-3c816f2b8f2d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.028356 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420ecf5-525c-42a0-91c4-3c816f2b8f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.049664 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8420ecf5-525c-42a0-91c4-3c816f2b8f2d None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:55.052050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8420ecf5-525c-42a0-91c4-3c816f2b8f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:55.127545 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 395/796] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:13:55.128495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420ecf5-525c-42a0-91c4-3c816f2b8f2d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:55.144644 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e509436-606b-4bd1-8342-64aa500e6b92 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:55.150456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:55.179855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5f55e-29ef-42aa-be46-d2bb708eb85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:55.180400 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5f55e-29ef-42aa-be46-d2bb708eb85d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.180678 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5f55e-29ef-42aa-be46-d2bb708eb85d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.180947 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5f55e-29ef-42aa-be46-d2bb708eb85d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.192548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5f55e-29ef-42aa-be46-d2bb708eb85d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:55.194140 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5f55e-29ef-42aa-be46-d2bb708eb85d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:55.208519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-28c5f55e-29ef-42aa-be46-d2bb708eb85d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:55.208519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:55.208519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:55.208519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:55.208519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:55.208519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:55.208519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:55.208519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:55.208519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:55.208519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:55.208519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:55.208519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:55.208519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:55.208519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:55.208519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:55.208519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:55.209452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:55.209452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:55.209452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:55.209452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:55.209452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:55.209452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:55.209452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:55.209452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:55.209452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:55.209452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:55.209452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:55.209452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:55.209452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:55.209452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:55.209452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:55.209452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:55.209452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:55.210966 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:55.210966 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:55.210966 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:55.210966 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 402/797] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:55.211674 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067137f3-e887-46b2-9eea-0414beedeb7a None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.212153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067137f3-e887-46b2-9eea-0414beedeb7a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.212438 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067137f3-e887-46b2-9eea-0414beedeb7a None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.217897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 403/798] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:55.217897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067137f3-e887-46b2-9eea-0414beedeb7a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:55.242107 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420ecf5-525c-42a0-91c4-3c816f2b8f2d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:55.266939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:55.278749 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b9079-24df-411d-ad25-2a36af17ecfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:55.279371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b9079-24df-411d-ad25-2a36af17ecfa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.279644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b9079-24df-411d-ad25-2a36af17ecfa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.279908 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b9079-24df-411d-ad25-2a36af17ecfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.287147 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b9079-24df-411d-ad25-2a36af17ecfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:55.288586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b9079-24df-411d-ad25-2a36af17ecfa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:55.299469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-360b9079-24df-411d-ad25-2a36af17ecfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:55.299469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:55.299469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:55.299469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:55.299469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:55.299469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:55.299469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:55.299469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:55.299469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:55.299469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:55.299469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:55.299469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:55.299469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:55.299469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:55.299469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:55.300346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:55.300346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:55.300346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:55.300346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:55.300346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:55.300346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:55.300346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:55.300346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:55.300346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:55.300346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:55.300346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:55.300346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:55.300346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:55.300346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:55.300346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:55.300346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:55.300346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:55.301788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:55.301788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:55.301788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:55.301788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:55.301788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:55.301788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:55.301788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:55.301788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:55.301788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 396/799] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:55.302946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b9079-24df-411d-ad25-2a36af17ecfa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:55.332137 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067137f3-e887-46b2-9eea-0414beedeb7a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:55.367447 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a17dda-f4f1-4ba0-be9c-cab887dd2154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:55.368068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a17dda-f4f1-4ba0-be9c-cab887dd2154 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.368353 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a17dda-f4f1-4ba0-be9c-cab887dd2154 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.368615 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a17dda-f4f1-4ba0-be9c-cab887dd2154 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.381158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a17dda-f4f1-4ba0-be9c-cab887dd2154 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:55.385818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-25a17dda-f4f1-4ba0-be9c-cab887dd2154 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:55.385818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:55.385818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:55.385818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:55.385818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:55.385818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:55.385818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:55.385818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:55.385818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:55.385818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:55.385818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:55.385818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:55.385818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:55.385818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:55.385818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:55.385818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:55.386854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:55.386854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:55.386854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:55.386854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:55.386854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:55.386854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:55.386854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:55.386854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:55.386854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:55.386854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:55.386854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:55.386854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:55.386854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:55.386854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:55.386854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:55.386854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:55.387698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:55.387698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:55.387698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:55.387698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:55.387698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:55.387698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:55.387698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:55.387698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:55.387698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:55.387698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:55.387698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:13:55.387698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:55.387698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 404/800] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:13:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:55.388498 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f0e6e-f09f-46b3-af1b-a915e027a63b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.388498 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f0e6e-f09f-46b3-af1b-a915e027a63b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.388619 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f0e6e-f09f-46b3-af1b-a915e027a63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.393342 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:55.424802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b9079-24df-411d-ad25-2a36af17ecfa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:55.425373 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-402f0e6e-f09f-46b3-af1b-a915e027a63b None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:55.427228 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-402f0e6e-f09f-46b3-af1b-a915e027a63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:55.467146 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b19ede-c44f-45cb-ac22-7c722468e3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:55.467763 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b19ede-c44f-45cb-ac22-7c722468e3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.468044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b19ede-c44f-45cb-ac22-7c722468e3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.468318 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b19ede-c44f-45cb-ac22-7c722468e3f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.468973 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b19ede-c44f-45cb-ac22-7c722468e3f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:55.469283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b19ede-c44f-45cb-ac22-7c722468e3f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:55.470692 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b19ede-c44f-45cb-ac22-7c722468e3f9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:55.487965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 397/801] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:55.487965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b19ede-c44f-45cb-ac22-7c722468e3f9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:55.508364 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 405/802] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:13:55.509139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f0e6e-f09f-46b3-af1b-a915e027a63b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:55.543714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:55.606865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b19ede-c44f-45cb-ac22-7c722468e3f9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:55.616052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f0e6e-f09f-46b3-af1b-a915e027a63b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:55.645354 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73683eb9-ac6d-44a2-9468-7c1ac6e41fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:55.646128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73683eb9-ac6d-44a2-9468-7c1ac6e41fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.646435 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73683eb9-ac6d-44a2-9468-7c1ac6e41fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.646706 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73683eb9-ac6d-44a2-9468-7c1ac6e41fa1 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.647390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73683eb9-ac6d-44a2-9468-7c1ac6e41fa1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:55.647691 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73683eb9-ac6d-44a2-9468-7c1ac6e41fa1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:55.649146 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73683eb9-ac6d-44a2-9468-7c1ac6e41fa1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:55.654440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed56bda-5eb5-40ee-ab0f-ff234c5208cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:55.655298 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed56bda-5eb5-40ee-ab0f-ff234c5208cf None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.656239 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:55.656607 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed56bda-5eb5-40ee-ab0f-ff234c5208cf None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.657016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed56bda-5eb5-40ee-ab0f-ff234c5208cf None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.658577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed56bda-5eb5-40ee-ab0f-ff234c5208cf None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:55.659299 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed56bda-5eb5-40ee-ab0f-ff234c5208cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:55.661457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed56bda-5eb5-40ee-ab0f-ff234c5208cf None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:55.667105 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 398/803] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:55.667955 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73683eb9-ac6d-44a2-9468-7c1ac6e41fa1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:55.675881 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 406/804] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:13:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:55.675881 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed56bda-5eb5-40ee-ab0f-ff234c5208cf None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:55.779148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:55.786095 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed56bda-5eb5-40ee-ab0f-ff234c5208cf None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:55.790085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73683eb9-ac6d-44a2-9468-7c1ac6e41fa1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:55.821446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61371c-7ab2-41a2-91f8-06aa23a46678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:55.822065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61371c-7ab2-41a2-91f8-06aa23a46678 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.822377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61371c-7ab2-41a2-91f8-06aa23a46678 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.822650 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61371c-7ab2-41a2-91f8-06aa23a46678 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.823226 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb61371c-7ab2-41a2-91f8-06aa23a46678 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:55.824594 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb61371c-7ab2-41a2-91f8-06aa23a46678 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:55.827692 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceabe9e-7b0f-4e38-86f3-47e47c95f4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:55.828543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceabe9e-7b0f-4e38-86f3-47e47c95f4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.829037 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceabe9e-7b0f-4e38-86f3-47e47c95f4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.829514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceabe9e-7b0f-4e38-86f3-47e47c95f4e6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.833722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-fb61371c-7ab2-41a2-91f8-06aa23a46678 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:13:55.833722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:55.833722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:13:55.833722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:13:55.833722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:13:55.833722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:13:55.833722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:13:55.833722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:13:55.833722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:13:55.833722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:55.833722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:13:55.833722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:55.833722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:55.833722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:55.833722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:55.833722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:55.833722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:55.834806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:55.834806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:55.834806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:55.834806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:55.834806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:55.834806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:55.834806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:13:55.834806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:13:55.834806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:13:55.834806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:13:55.834806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:55.834806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:55.834806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:55.834806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:55.834806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:55.834806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:55.835655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:55.835655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:55.835655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:55.835655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:55.835655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:55.835655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:55.835655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:55.835655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:55.835655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:55.835655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:55.835655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:13:55.835655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:13:55.835655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:13:55.835655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:55.835655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 407/805] 10.222.0.39 () {56 vars in 1129 bytes} [Mon Aug 8 02:13:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:55.836427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61371c-7ab2-41a2-91f8-06aa23a46678 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:55.841152 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceabe9e-7b0f-4e38-86f3-47e47c95f4e6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:55.842614 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceabe9e-7b0f-4e38-86f3-47e47c95f4e6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:55.854491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5ceabe9e-7b0f-4e38-86f3-47e47c95f4e6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:13:55.854491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:55.854491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:55.854491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:55.854491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:55.854491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:55.854491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:55.854491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:55.854491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:55.854491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:55.854491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:55.854491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:55.854491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:55.854491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:55.854491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:55.854491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:55.855456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:55.855456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:55.855456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:55.855456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:55.855456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:55.855456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:55.855456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:55.855456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:55.855456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:55.855456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:55.855456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:55.855456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:55.855456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:55.855456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:55.855456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:55.855456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:55.855456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:55.855456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:55.856392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:55.856392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:55.856392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:55.856392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:55.856392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:55.856392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:55.856392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:55.856392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:55.856392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:13:55.856392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:55.856392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 399/806] 10.222.0.39 () {56 vars in 1093 bytes} [Mon Aug 8 02:13:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:55.856392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceabe9e-7b0f-4e38-86f3-47e47c95f4e6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:55.916892 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:55.945048 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61371c-7ab2-41a2-91f8-06aa23a46678 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:55.958308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceabe9e-7b0f-4e38-86f3-47e47c95f4e6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:55.977909 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cba1e-37ac-412b-947e-91582e826eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:55.978590 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cba1e-37ac-412b-947e-91582e826eb4 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.978866 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cba1e-37ac-412b-947e-91582e826eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.979136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cba1e-37ac-412b-947e-91582e826eb4 None admin] PATH_INFO: `/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/ac60b4d621234225830c94edfac400e2/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.980999 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cba1e-37ac-412b-947e-91582e826eb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d474b61f1fe4f17aacb7a3665315c79, user_id=ac60b4d621234225830c94edfac400e2, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:55.982441 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cba1e-37ac-412b-947e-91582e826eb4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:55.992979 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b72a3e-883b-43ea-a468-7ff9e124b666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:55.993590 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b72a3e-883b-43ea-a468-7ff9e124b666 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.993863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b72a3e-883b-43ea-a468-7ff9e124b666 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.994130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b72a3e-883b-43ea-a468-7ff9e124b666 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.994648 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 408/807] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:13:55 2022] PUT /identity/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/ac60b4d621234225830c94edfac400e2/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:55.995428 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cba1e-37ac-412b-947e-91582e826eb4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.020792 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b72a3e-883b-43ea-a468-7ff9e124b666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:56.022315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b72a3e-883b-43ea-a468-7ff9e124b666 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:56.023230 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:56.038741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-36b72a3e-883b-43ea-a468-7ff9e124b666 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:56.038741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:56.038741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:56.038741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:56.038741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:56.038741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:56.038741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:56.038741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:56.038741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:56.038741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:56.038741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:56.038741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:56.038741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:56.038741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:56.038741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:56.038741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:56.039857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:56.039857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:56.039857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:56.039857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:56.039857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:56.039857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:56.039857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:56.039857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:56.039857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:56.039857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:56.039857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:56.039857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:56.039857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:56.039857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:56.039857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:56.039857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:56.039857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:56.040765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:56.040765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:56.040765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:56.040765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 400/808] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:56.041617 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0f368-dffb-49d4-bc57-1d0dfb4d4312 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.041727 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0f368-dffb-49d4-bc57-1d0dfb4d4312 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.041987 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0f368-dffb-49d4-bc57-1d0dfb4d4312 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.043511 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 401/809] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:56.044158 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0f368-dffb-49d4-bc57-1d0dfb4d4312 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.126372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cba1e-37ac-412b-947e-91582e826eb4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:56.166066 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0f368-dffb-49d4-bc57-1d0dfb4d4312 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:56.169996 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c9022-021a-4247-a86e-f3e15b235930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:56.174318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c9022-021a-4247-a86e-f3e15b235930 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.174318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c9022-021a-4247-a86e-f3e15b235930 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.174318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c9022-021a-4247-a86e-f3e15b235930 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.174318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c9022-021a-4247-a86e-f3e15b235930 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:56.174318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c9022-021a-4247-a86e-f3e15b235930 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:56.174318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c9022-021a-4247-a86e-f3e15b235930 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:56.177668 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:56.179588 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 409/810] 10.222.0.39 () {56 vars in 1124 bytes} [Mon Aug 8 02:13:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:56.180423 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c9022-021a-4247-a86e-f3e15b235930 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.208466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43349f3-348e-4587-8d7c-309c0f6b3e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:56.209072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43349f3-348e-4587-8d7c-309c0f6b3e9b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.209351 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43349f3-348e-4587-8d7c-309c0f6b3e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.209620 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43349f3-348e-4587-8d7c-309c0f6b3e9b None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.210379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43349f3-348e-4587-8d7c-309c0f6b3e9b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:56.210687 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43349f3-348e-4587-8d7c-309c0f6b3e9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:56.212163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43349f3-348e-4587-8d7c-309c0f6b3e9b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:56.229254 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 402/811] 10.222.0.39 () {56 vars in 1133 bytes} [Mon Aug 8 02:13:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:56.234298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f59f0-e41c-4bbe-a3ca-fd664de6329c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.234298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f59f0-e41c-4bbe-a3ca-fd664de6329c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.234298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f59f0-e41c-4bbe-a3ca-fd664de6329c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.248756 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9a2f59f0-e41c-4bbe-a3ca-fd664de6329c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:56.249941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9a2f59f0-e41c-4bbe-a3ca-fd664de6329c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:56.280168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:56.299708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c9022-021a-4247-a86e-f3e15b235930 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:56.329405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 403/812] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:13:56.329405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f59f0-e41c-4bbe-a3ca-fd664de6329c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.351820 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ed148-7452-47a0-8f9e-a86db742feca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:56.352401 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ed148-7452-47a0-8f9e-a86db742feca None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.352675 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ed148-7452-47a0-8f9e-a86db742feca None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.352943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ed148-7452-47a0-8f9e-a86db742feca None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.353586 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ed148-7452-47a0-8f9e-a86db742feca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:56.353889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ed148-7452-47a0-8f9e-a86db742feca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:56.355350 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ed148-7452-47a0-8f9e-a86db742feca None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:56.372263 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 410/813] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:56.372263 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ed148-7452-47a0-8f9e-a86db742feca None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.432226 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:56.439104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f59f0-e41c-4bbe-a3ca-fd664de6329c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:56.478287 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1861d-ffaf-4163-93d6-a4955668fbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:56.478897 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1861d-ffaf-4163-93d6-a4955668fbdc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.479170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1861d-ffaf-4163-93d6-a4955668fbdc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.479436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1861d-ffaf-4163-93d6-a4955668fbdc None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.480054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1861d-ffaf-4163-93d6-a4955668fbdc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:56.480352 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1861d-ffaf-4163-93d6-a4955668fbdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:56.481713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1861d-ffaf-4163-93d6-a4955668fbdc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:56.486975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 404/814] 10.222.0.39 () {56 vars in 1124 bytes} [Mon Aug 8 02:13:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:56.487815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1861d-ffaf-4163-93d6-a4955668fbdc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.492562 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ed148-7452-47a0-8f9e-a86db742feca None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:56.529080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92a47b-fd3b-4503-8e50-76121e364a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:56.529814 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92a47b-fd3b-4503-8e50-76121e364a1f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.530092 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92a47b-fd3b-4503-8e50-76121e364a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.530392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92a47b-fd3b-4503-8e50-76121e364a1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.546376 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:56.593208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1861d-ffaf-4163-93d6-a4955668fbdc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:56.608171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b92a47b-fd3b-4503-8e50-76121e364a1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:56.609604 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b92a47b-fd3b-4503-8e50-76121e364a1f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:56.628305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5e851-deb8-4cda-afc3-e3467546f847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:56.628951 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5e851-deb8-4cda-afc3-e3467546f847 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.629224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5e851-deb8-4cda-afc3-e3467546f847 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.629491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5e851-deb8-4cda-afc3-e3467546f847 None admin] PATH_INFO: `/v3/services/network` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.630065 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5e851-deb8-4cda-afc3-e3467546f847 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:56.631444 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5e851-deb8-4cda-afc3-e3467546f847 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:56.640127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7ce5e851-deb8-4cda-afc3-e3467546f847 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:13:56.640127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:56.640127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:13:56.640127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:13:56.640127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:13:56.640127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:13:56.640127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:13:56.640127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:13:56.640127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:13:56.640127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:56.640127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:13:56.640127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:56.640127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:56.640127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:56.640127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:56.640127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:56.640127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:56.641313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:56.641313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:56.641313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:56.641313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:56.641313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:56.641313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:56.641313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:13:56.641313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:13:56.641313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:13:56.641313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:13:56.641313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:56.641313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:56.641313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:56.641313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:56.641313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:56.641313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:56.642390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:56.642390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:56.642390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:56.642390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:56.642390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:56.642390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:56.642390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:56.642390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:56.642390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:56.642390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:56.642390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:13:56.642390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:13:56.642390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:13:56.642390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:56.646381 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 405/815] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:56 2022] GET /identity/v3/services/network => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:56.647243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5e851-deb8-4cda-afc3-e3467546f847 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.675789 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:56.716882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 411/816] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:13:56.722104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92a47b-fd3b-4503-8e50-76121e364a1f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.755109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5e851-deb8-4cda-afc3-e3467546f847 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:56.768846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:56.795281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291dc361-ea65-4097-8604-ac2058cb81c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:56.796702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dc361-ea65-4097-8604-ac2058cb81c7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.796975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dc361-ea65-4097-8604-ac2058cb81c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.797241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dc361-ea65-4097-8604-ac2058cb81c7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.805029 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291dc361-ea65-4097-8604-ac2058cb81c7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:56.806501 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291dc361-ea65-4097-8604-ac2058cb81c7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:56.818335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-291dc361-ea65-4097-8604-ac2058cb81c7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:13:56.818335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:56.818335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:56.818335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:56.818335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:56.818335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:56.818335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:56.818335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:56.818335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:56.818335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:56.818335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:56.818335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:56.818335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:56.818335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:56.818335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:56.818335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:56.819243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:56.819243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:56.819243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:56.819243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:56.819243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:56.819243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:56.819243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:56.819243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:56.819243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:56.819243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:56.819243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:56.819243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:56.819243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:56.819243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:56.819243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:56.819243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:56.819243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:56.819243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:56.820170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:56.820170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:56.820170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:56.820170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:56.820170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:56.820170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:56.820170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:56.820170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:56.820170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:13:56.820170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:56.820170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 406/817] 10.222.0.39 () {56 vars in 1117 bytes} [Mon Aug 8 02:13:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:56.820170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291dc361-ea65-4097-8604-ac2058cb81c7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.827851 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92a47b-fd3b-4503-8e50-76121e364a1f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:56.866652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a11ec-6e7b-46cc-aa04-7e65621fb37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:56.867540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a11ec-6e7b-46cc-aa04-7e65621fb37b None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.868016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a11ec-6e7b-46cc-aa04-7e65621fb37b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.868795 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a11ec-6e7b-46cc-aa04-7e65621fb37b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.869948 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a11ec-6e7b-46cc-aa04-7e65621fb37b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:56.871852 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a11ec-6e7b-46cc-aa04-7e65621fb37b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:56.896100 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 412/818] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:13:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:56.897274 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a11ec-6e7b-46cc-aa04-7e65621fb37b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.926498 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:56.945702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291dc361-ea65-4097-8604-ac2058cb81c7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:56.990211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0446e2-9157-493a-b76c-b26e54537d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:56.990859 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0446e2-9157-493a-b76c-b26e54537d43 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.991157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0446e2-9157-493a-b76c-b26e54537d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.991520 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0446e2-9157-493a-b76c-b26e54537d43 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.992162 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0446e2-9157-493a-b76c-b26e54537d43 None admin] RBAC: Adding query filter params (name=network) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:56.992468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0446e2-9157-493a-b76c-b26e54537d43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:56.993938 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0446e2-9157-493a-b76c-b26e54537d43 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:57.004335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 407/819] 10.222.0.39 () {56 vars in 1112 bytes} [Mon Aug 8 02:13:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:57.004335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0446e2-9157-493a-b76c-b26e54537d43 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:57.033373 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:57.037780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a11ec-6e7b-46cc-aa04-7e65621fb37b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:57.085332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3df1a4-713a-4cb6-92ef-113283009e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:57.085974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3df1a4-713a-4cb6-92ef-113283009e60 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:57.086274 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3df1a4-713a-4cb6-92ef-113283009e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:57.086559 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3df1a4-713a-4cb6-92ef-113283009e60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:57.102555 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3df1a4-713a-4cb6-92ef-113283009e60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:57.104779 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3df1a4-713a-4cb6-92ef-113283009e60 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:57.123251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-dc3df1a4-713a-4cb6-92ef-113283009e60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:57.123251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:57.123251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:57.123251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:57.123251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:57.123251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:57.123251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:57.123251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:57.123251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:57.123251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:57.123251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:57.123251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:57.123251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:57.123251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:57.123251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:57.123251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:57.124203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:57.124203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:57.124203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:57.124203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:57.124203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:57.124203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:57.124203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:57.124203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:57.124203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:57.124203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:57.124203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:57.124203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:57.124203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:57.124203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:57.124203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:57.124203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:57.124203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:57.125071 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:57.125071 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:57.125071 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:57.125071 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 413/820] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:57.125779 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3df1a4-713a-4cb6-92ef-113283009e60 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:57.149753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0446e2-9157-493a-b76c-b26e54537d43 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:57.196025 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906a7c0-ea57-4849-bbac-675e2368d693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:57.197100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906a7c0-ea57-4849-bbac-675e2368d693 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:57.197504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906a7c0-ea57-4849-bbac-675e2368d693 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:57.197824 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906a7c0-ea57-4849-bbac-675e2368d693 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:57.198752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906a7c0-ea57-4849-bbac-675e2368d693 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:57.199062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906a7c0-ea57-4849-bbac-675e2368d693 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:57.201110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906a7c0-ea57-4849-bbac-675e2368d693 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:57.224345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 408/821] 10.222.0.39 () {56 vars in 1149 bytes} [Mon Aug 8 02:13:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:57.225337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906a7c0-ea57-4849-bbac-675e2368d693 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:57.239141 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:57.245432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3df1a4-713a-4cb6-92ef-113283009e60 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:57.287045 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865531b1-7d00-41e5-a33e-4893c46d8ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:57.287780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865531b1-7d00-41e5-a33e-4893c46d8ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:57.288071 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865531b1-7d00-41e5-a33e-4893c46d8ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:57.288351 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865531b1-7d00-41e5-a33e-4893c46d8ce1 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:57.289153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865531b1-7d00-41e5-a33e-4893c46d8ce1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:57.289465 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865531b1-7d00-41e5-a33e-4893c46d8ce1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:57.291170 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865531b1-7d00-41e5-a33e-4893c46d8ce1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:57.297468 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 414/822] 10.222.0.39 () {56 vars in 1112 bytes} [Mon Aug 8 02:13:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:57.302730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865531b1-7d00-41e5-a33e-4893c46d8ce1 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:57.329901 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:57.351969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906a7c0-ea57-4849-bbac-675e2368d693 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:57.390380 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e6cac-18a6-480c-895f-9648cf7177dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:57.391547 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e6cac-18a6-480c-895f-9648cf7177dd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:57.392149 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e6cac-18a6-480c-895f-9648cf7177dd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:57.392802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e6cac-18a6-480c-895f-9648cf7177dd None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:57.394073 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e6cac-18a6-480c-895f-9648cf7177dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:57.394923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e6cac-18a6-480c-895f-9648cf7177dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:57.396620 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e6cac-18a6-480c-895f-9648cf7177dd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:57.409435 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 409/823] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:57.410451 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e6cac-18a6-480c-895f-9648cf7177dd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:57.446463 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865531b1-7d00-41e5-a33e-4893c46d8ce1 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:57.490531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ccaf1-9831-48e2-aa85-d5d3ffb5c9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:57.491165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ccaf1-9831-48e2-aa85-d5d3ffb5c9eb None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:57.491439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ccaf1-9831-48e2-aa85-d5d3ffb5c9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:57.491709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ccaf1-9831-48e2-aa85-d5d3ffb5c9eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:57.502282 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:57.506882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ccaf1-9831-48e2-aa85-d5d3ffb5c9eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:57.509862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ccaf1-9831-48e2-aa85-d5d3ffb5c9eb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:57.528475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-960ccaf1-9831-48e2-aa85-d5d3ffb5c9eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:57.528475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:57.528475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:57.528475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:57.528475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:57.528475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:57.528475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:57.528475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:57.528475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:57.528475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:57.528475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:57.528475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:57.528475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:57.528475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:57.528475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:57.528475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:57.529477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:57.529477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:57.529477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:57.529477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:57.529477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:57.529477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:57.529477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:57.529477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:57.529477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:57.529477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:57.529477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:57.529477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:57.529477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:57.529477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:57.529477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:57.529477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:57.529477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:57.530458 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:57.530458 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:57.530458 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:57.534314 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 415/824] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:57.538562 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ccaf1-9831-48e2-aa85-d5d3ffb5c9eb None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:57.552778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e6cac-18a6-480c-895f-9648cf7177dd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:57.588538 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dcb0c8-90e4-4171-b93e-d38368bed50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:57.589145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dcb0c8-90e4-4171-b93e-d38368bed50a None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:57.589421 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dcb0c8-90e4-4171-b93e-d38368bed50a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:57.589689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dcb0c8-90e4-4171-b93e-d38368bed50a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:57.590363 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dcb0c8-90e4-4171-b93e-d38368bed50a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:57.591719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dcb0c8-90e4-4171-b93e-d38368bed50a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:57.622265 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 410/825] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:13:57 2022] POST /identity/v3/endpoints => generated 337 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:57.622265 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dcb0c8-90e4-4171-b93e-d38368bed50a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:57.643897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:57.736908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ccaf1-9831-48e2-aa85-d5d3ffb5c9eb None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:57.826379 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02614668-892c-4f1f-ac82-f184fe917520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:57.827004 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02614668-892c-4f1f-ac82-f184fe917520 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:57.827274 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02614668-892c-4f1f-ac82-f184fe917520 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:57.827538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02614668-892c-4f1f-ac82-f184fe917520 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:57.833028 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dcb0c8-90e4-4171-b93e-d38368bed50a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:57.839697 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02614668-892c-4f1f-ac82-f184fe917520 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:57.841742 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02614668-892c-4f1f-ac82-f184fe917520 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:57.857869 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:57.862161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-02614668-892c-4f1f-ac82-f184fe917520 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:57.862161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:57.862161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:57.862161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:57.862161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:57.862161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:57.862161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:57.862161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:57.862161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:57.862161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:57.862161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:57.862161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:57.862161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:57.862161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:57.862161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:57.863063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:57.863063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:57.863063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:57.863063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:57.863063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:57.863063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:57.863063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:57.863063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:57.863063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:57.863063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:57.863063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:57.863063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:57.863063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:57.863063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:57.863063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:57.863063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:57.863063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:57.864629 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:57.864629 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:57.864629 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:57.864629 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:57.864629 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:57.864629 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:57.864629 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:57.864629 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:57.872925 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 416/826] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:57.874947 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02614668-892c-4f1f-ac82-f184fe917520 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:57.889515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb9a25-ae90-4d37-96d9-cde23022e87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:57.890430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb9a25-ae90-4d37-96d9-cde23022e87a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:57.890691 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb9a25-ae90-4d37-96d9-cde23022e87a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:57.891053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb9a25-ae90-4d37-96d9-cde23022e87a None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:57.891811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb9a25-ae90-4d37-96d9-cde23022e87a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:57.892212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb9a25-ae90-4d37-96d9-cde23022e87a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:57.893666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb9a25-ae90-4d37-96d9-cde23022e87a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:57.908604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 411/827] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:57.914626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb9a25-ae90-4d37-96d9-cde23022e87a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:57.998441 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02614668-892c-4f1f-ac82-f184fe917520 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:58.027988 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:58.043252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559cdd6-947c-4782-9e1a-9b047eb055eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:58.043863 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559cdd6-947c-4782-9e1a-9b047eb055eb None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:58.044159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559cdd6-947c-4782-9e1a-9b047eb055eb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:58.044422 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559cdd6-947c-4782-9e1a-9b047eb055eb None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.045122 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4559cdd6-947c-4782-9e1a-9b047eb055eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:58.045645 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4559cdd6-947c-4782-9e1a-9b047eb055eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:58.046932 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4559cdd6-947c-4782-9e1a-9b047eb055eb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:58.048648 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb9a25-ae90-4d37-96d9-cde23022e87a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:58.068269 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 417/828] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:58.069418 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a8224-202b-4757-b56b-cd011734eed0 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:58.069766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a8224-202b-4757-b56b-cd011734eed0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:58.070043 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a8224-202b-4757-b56b-cd011734eed0 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.071502 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 418/829] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:58.078016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f54e42-62c2-40b9-b267-a4218dc32a36 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:58.078335 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f54e42-62c2-40b9-b267-a4218dc32a36 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:58.078700 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f54e42-62c2-40b9-b267-a4218dc32a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.092689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f812925-3e05-41bc-9d0c-e9abf2161f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:58.093523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f812925-3e05-41bc-9d0c-e9abf2161f2d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:58.093968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f812925-3e05-41bc-9d0c-e9abf2161f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:58.094407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f812925-3e05-41bc-9d0c-e9abf2161f2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.096930 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-46f54e42-62c2-40b9-b267-a4218dc32a36 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:58.098179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-46f54e42-62c2-40b9-b267-a4218dc32a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:58.101842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f812925-3e05-41bc-9d0c-e9abf2161f2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:58.103560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f812925-3e05-41bc-9d0c-e9abf2161f2d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:58.117166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1f812925-3e05-41bc-9d0c-e9abf2161f2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:58.117166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:58.117166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:58.117166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:58.117166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:58.117166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:58.117166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:58.117166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:58.117166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:58.117166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:58.117166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:58.117166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:58.117166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:13:58.117166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:13:58.117166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:13:58.118221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:13:58.118221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:58.118221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:58.118221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:58.118221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:58.118221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:58.118221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:58.118221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:58.118221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:58.118221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:58.118221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:58.118221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:58.118221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:58.118221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:58.118221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:58.118221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:58.118221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:58.119332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:13:58.119332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:13:58.119332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:13:58.119332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:13:58.119332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:13:58.119332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:13:58.119332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:13:58.119332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:58.121655 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 412/830] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:58.122807 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f812925-3e05-41bc-9d0c-e9abf2161f2d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:58.165942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:58.171155 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 419/831] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:13:58.171904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f54e42-62c2-40b9-b267-a4218dc32a36 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:58.246404 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f812925-3e05-41bc-9d0c-e9abf2161f2d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:58.277597 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f54e42-62c2-40b9-b267-a4218dc32a36 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:58.281047 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac321359-737c-4690-9eda-0525c48af06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:58.281635 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac321359-737c-4690-9eda-0525c48af06c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:58.281908 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac321359-737c-4690-9eda-0525c48af06c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:58.282173 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac321359-737c-4690-9eda-0525c48af06c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.282933 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac321359-737c-4690-9eda-0525c48af06c None admin] RBAC: Adding query filter params (role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=c4238566a9c248ffbfa314f47f02670c) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:58.283237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac321359-737c-4690-9eda-0525c48af06c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:58.284651 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac321359-737c-4690-9eda-0525c48af06c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:58.292171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 413/832] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:58 2022] GET /identity/v3/role_assignments?user.id=c4238566a9c248ffbfa314f47f02670c&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:58.293176 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac321359-737c-4690-9eda-0525c48af06c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:58.304069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:58.338778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989007b-c257-42a6-9711-7f4bd90a4e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:58.339423 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989007b-c257-42a6-9711-7f4bd90a4e1c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:58.339698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989007b-c257-42a6-9711-7f4bd90a4e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:58.339966 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989007b-c257-42a6-9711-7f4bd90a4e1c None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.340666 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989007b-c257-42a6-9711-7f4bd90a4e1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:58.340972 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989007b-c257-42a6-9711-7f4bd90a4e1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:58.343125 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989007b-c257-42a6-9711-7f4bd90a4e1c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:58.364560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 420/833] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:13:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:58.371004 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989007b-c257-42a6-9711-7f4bd90a4e1c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:58.422849 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:58.426727 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac321359-737c-4690-9eda-0525c48af06c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:58.478698 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c4642-823f-43e3-882e-3b92667bc7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:58.479341 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c4642-823f-43e3-882e-3b92667bc7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:58.479620 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c4642-823f-43e3-882e-3b92667bc7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:58.479890 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c4642-823f-43e3-882e-3b92667bc7d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.495879 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c4642-823f-43e3-882e-3b92667bc7d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:58.503822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b74c4642-823f-43e3-882e-3b92667bc7d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:58.503822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:58.503822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:58.503822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:58.503822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:58.503822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:58.503822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:58.503822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:58.503822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:58.503822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:58.503822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:58.503822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:58.503822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:58.503822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:58.503822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:58.503822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:58.504946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:58.504946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:58.504946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:58.504946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:58.504946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:58.504946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:58.504946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:58.504946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:58.504946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:58.504946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:58.504946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:58.504946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:58.504946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:58.504946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:58.504946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:58.504946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:58.505978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:58.505978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:58.505978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:58.505978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:58.505978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:58.505978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:58.505978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:58.505978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:58.505978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:58.505978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:58.505978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:58.505978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:58.506681 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 414/834] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:58.513722 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c4642-823f-43e3-882e-3b92667bc7d3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:58.515071 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989007b-c257-42a6-9711-7f4bd90a4e1c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:58.562857 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eaab10-e2d5-43d1-a362-fc645b3b3e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:58.563619 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eaab10-e2d5-43d1-a362-fc645b3b3e70 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:58.563952 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eaab10-e2d5-43d1-a362-fc645b3b3e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:58.564287 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eaab10-e2d5-43d1-a362-fc645b3b3e70 None admin] PATH_INFO: `/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/b88ccd3e5c4c48d3a27f4eaa7874e28e/roles/838b081e674f4c0ab4a33bcf8e614dc5` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.583919 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eaab10-e2d5-43d1-a362-fc645b3b3e70 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d474b61f1fe4f17aacb7a3665315c79, user_id=b88ccd3e5c4c48d3a27f4eaa7874e28e, role_id=838b081e674f4c0ab4a33bcf8e614dc5)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:58.586156 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eaab10-e2d5-43d1-a362-fc645b3b3e70 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:58.611733 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 421/835] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:13:58 2022] PUT /identity/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/b88ccd3e5c4c48d3a27f4eaa7874e28e/roles/838b081e674f4c0ab4a33bcf8e614dc5 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:58.643309 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c4642-823f-43e3-882e-3b92667bc7d3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:58.685179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf9772-c30e-4443-89d8-75cb07834aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:58.685767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf9772-c30e-4443-89d8-75cb07834aff None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:58.686047 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf9772-c30e-4443-89d8-75cb07834aff None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:58.686337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf9772-c30e-4443-89d8-75cb07834aff None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.687071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf9772-c30e-4443-89d8-75cb07834aff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:58.687382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf9772-c30e-4443-89d8-75cb07834aff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:58.688888 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf9772-c30e-4443-89d8-75cb07834aff None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:58.699373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 415/836] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:58.705050 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eaab10-e2d5-43d1-a362-fc645b3b3e70 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:58.736162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:58.820059 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eaab10-e2d5-43d1-a362-fc645b3b3e70 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:58.857777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a48c88-f6bc-40a3-8f31-028ea325a3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:58.858434 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a48c88-f6bc-40a3-8f31-028ea325a3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:58.858718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a48c88-f6bc-40a3-8f31-028ea325a3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:58.858989 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a48c88-f6bc-40a3-8f31-028ea325a3d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.869748 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a48c88-f6bc-40a3-8f31-028ea325a3d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:58.871219 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a48c88-f6bc-40a3-8f31-028ea325a3d6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:58.886070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-28a48c88-f6bc-40a3-8f31-028ea325a3d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:58.886070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:58.886070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:58.886070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:58.886070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:58.886070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:58.886070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:58.886070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:58.886070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:58.886070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:58.886070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:58.886070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:58.886070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:58.886070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:58.886070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:58.886070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:58.887106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:58.887106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:58.887106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:58.887106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:58.887106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:58.887106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:58.887106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:58.887106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:58.887106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:58.887106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:58.887106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:58.887106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:58.887106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:58.887106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:58.887106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:58.887106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:58.887106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:58.887984 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:58.887984 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:58.887984 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:58.887984 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 422/837] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:58.893134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf9772-c30e-4443-89d8-75cb07834aff None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:58.920214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:59.001322 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf9772-c30e-4443-89d8-75cb07834aff None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:59.036949 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc0fd7a-8c7d-4e2e-8f53-806938b046ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:59.037550 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0fd7a-8c7d-4e2e-8f53-806938b046ab None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:59.037824 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0fd7a-8c7d-4e2e-8f53-806938b046ab None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:59.038091 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0fd7a-8c7d-4e2e-8f53-806938b046ab None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:59.038755 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc0fd7a-8c7d-4e2e-8f53-806938b046ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:59.039055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc0fd7a-8c7d-4e2e-8f53-806938b046ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:59.040413 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc0fd7a-8c7d-4e2e-8f53-806938b046ab None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:59.052945 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 416/838] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:59.058156 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a48c88-f6bc-40a3-8f31-028ea325a3d6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:59.087945 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed378a-3fa8-4ea0-bd30-ce9f3fc1a245 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:59.088276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed378a-3fa8-4ea0-bd30-ce9f3fc1a245 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:59.088515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed378a-3fa8-4ea0-bd30-ce9f3fc1a245 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:59.090098 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 417/839] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:59.096980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70af73-cd0f-4702-857d-e7b6b5b79aef None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:59.097454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70af73-cd0f-4702-857d-e7b6b5b79aef None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:59.097915 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70af73-cd0f-4702-857d-e7b6b5b79aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:59.115893 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9d70af73-cd0f-4702-857d-e7b6b5b79aef None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:59.117291 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9d70af73-cd0f-4702-857d-e7b6b5b79aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:59.158618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:59.185843 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a48c88-f6bc-40a3-8f31-028ea325a3d6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:59.206492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 418/840] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:13:59.227599 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70af73-cd0f-4702-857d-e7b6b5b79aef None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:59.235559 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38680e4-ddcd-4064-8eb8-b9a4422513ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:59.236432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38680e4-ddcd-4064-8eb8-b9a4422513ee None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:59.236907 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38680e4-ddcd-4064-8eb8-b9a4422513ee None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:59.237393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38680e4-ddcd-4064-8eb8-b9a4422513ee None admin] PATH_INFO: `/v3/users/swift` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:59.242531 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:59.247518 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38680e4-ddcd-4064-8eb8-b9a4422513ee None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:59.249173 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38680e4-ddcd-4064-8eb8-b9a4422513ee None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:59.262023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-d38680e4-ddcd-4064-8eb8-b9a4422513ee None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:13:59.262023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:59.262023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:59.262023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:59.262023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:59.262023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:59.262023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:59.262023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:59.262023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:59.262023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:59.262023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:59.262023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:59.262023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:13:59.262023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:13:59.262023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:13:59.262023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:13:59.263278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:59.263278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:59.263278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:13:59.263278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:59.263278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:13:59.263278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:13:59.263278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:59.263278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:59.263278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:59.263278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:59.263278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:59.263278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:59.263278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:59.263278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:59.263278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:59.263278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:59.263278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:59.263278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:59.264462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:59.264462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:59.264462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:13:59.264462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:13:59.264462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:13:59.264462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:13:59.264462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:13:59.264462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:13:59.264462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:13:59.264462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:13:59.270271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 423/841] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:59.271527 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38680e4-ddcd-4064-8eb8-b9a4422513ee None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:59.334696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70af73-cd0f-4702-857d-e7b6b5b79aef None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:59.373986 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91af140-e899-4931-a494-b6c8ca2e15d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:59.374652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91af140-e899-4931-a494-b6c8ca2e15d4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:59.375100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91af140-e899-4931-a494-b6c8ca2e15d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:59.375370 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91af140-e899-4931-a494-b6c8ca2e15d4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:59.375953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91af140-e899-4931-a494-b6c8ca2e15d4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:59.377354 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91af140-e899-4931-a494-b6c8ca2e15d4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:59.378336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38680e4-ddcd-4064-8eb8-b9a4422513ee None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:59.386466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b91af140-e899-4931-a494-b6c8ca2e15d4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:13:59.386466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:59.386466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:13:59.386466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:13:59.386466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:13:59.386466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:13:59.386466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:13:59.386466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:13:59.386466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:13:59.386466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:59.386466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:13:59.386466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:59.386466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:59.386466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:59.386466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:59.386466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:59.386466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:59.387663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:59.387663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:59.387663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:59.387663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:59.387663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:59.387663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:59.387663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:13:59.387663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:13:59.387663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:13:59.387663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:13:59.387663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:59.387663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:59.387663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:59.387663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:59.387663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:59.387663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:59.388716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:59.388716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:59.388716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:59.388716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:59.388716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:59.388716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:59.388716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:59.388716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:59.388716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:59.388716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:59.388716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:13:59.388716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:13:59.388716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:13:59.388716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:13:59.392495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 419/842] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:13:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:59.393880 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91af140-e899-4931-a494-b6c8ca2e15d4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:59.424182 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0a2e1-9f4c-49b2-a84d-075115ff602b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:59.424956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0a2e1-9f4c-49b2-a84d-075115ff602b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:59.425325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0a2e1-9f4c-49b2-a84d-075115ff602b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:59.425828 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0a2e1-9f4c-49b2-a84d-075115ff602b None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:59.426788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0a2e1-9f4c-49b2-a84d-075115ff602b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:59.427195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0a2e1-9f4c-49b2-a84d-075115ff602b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:59.428794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0a2e1-9f4c-49b2-a84d-075115ff602b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:59.432446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:59.458562 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 424/843] 10.222.0.39 () {56 vars in 1135 bytes} [Mon Aug 8 02:13:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:59.460341 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0a2e1-9f4c-49b2-a84d-075115ff602b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:59.493267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:59.528299 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91af140-e899-4931-a494-b6c8ca2e15d4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:59.569036 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218d6b7-a6b0-4879-a608-0f9e17d6c511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:59.569971 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218d6b7-a6b0-4879-a608-0f9e17d6c511 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:59.570117 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218d6b7-a6b0-4879-a608-0f9e17d6c511 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:59.570522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218d6b7-a6b0-4879-a608-0f9e17d6c511 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:59.571239 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218d6b7-a6b0-4879-a608-0f9e17d6c511 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:59.571637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218d6b7-a6b0-4879-a608-0f9e17d6c511 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:59.573158 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218d6b7-a6b0-4879-a608-0f9e17d6c511 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:59.578307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 420/844] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:13:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:59.584637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218d6b7-a6b0-4879-a608-0f9e17d6c511 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:59.587662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0a2e1-9f4c-49b2-a84d-075115ff602b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:59.630962 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c7a12-e92c-4336-ae06-9bba6efda9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:59.631570 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c7a12-e92c-4336-ae06-9bba6efda9ea None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:59.631848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c7a12-e92c-4336-ae06-9bba6efda9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:59.632118 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c7a12-e92c-4336-ae06-9bba6efda9ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:59.693804 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218d6b7-a6b0-4879-a608-0f9e17d6c511 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:59.712809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:59.722445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c7a12-e92c-4336-ae06-9bba6efda9ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:59.724993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c7a12-e92c-4336-ae06-9bba6efda9ea None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:59.731071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52878e94-6dcd-46fb-92d1-7ed5427eb063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:59.731976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52878e94-6dcd-46fb-92d1-7ed5427eb063 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:59.732447 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52878e94-6dcd-46fb-92d1-7ed5427eb063 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:59.732911 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52878e94-6dcd-46fb-92d1-7ed5427eb063 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:59.733793 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52878e94-6dcd-46fb-92d1-7ed5427eb063 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:59.734308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52878e94-6dcd-46fb-92d1-7ed5427eb063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:59.735841 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52878e94-6dcd-46fb-92d1-7ed5427eb063 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:59.741194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 421/845] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:13:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:59.848744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 425/846] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 02:13:59.855224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52878e94-6dcd-46fb-92d1-7ed5427eb063 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:59.859559 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:59.964743 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52878e94-6dcd-46fb-92d1-7ed5427eb063 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:59.998530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf82e4-c072-42d8-90ca-81d52521c3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:59.999525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf82e4-c072-42d8-90ca-81d52521c3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:59.999996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf82e4-c072-42d8-90ca-81d52521c3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:00.000488 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf82e4-c072-42d8-90ca-81d52521c3f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:00.011917 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf82e4-c072-42d8-90ca-81d52521c3f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:00.013701 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf82e4-c072-42d8-90ca-81d52521c3f4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:00.028820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-53cf82e4-c072-42d8-90ca-81d52521c3f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:00.028820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:00.028820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:00.028820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:00.028820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:00.028820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:00.028820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:00.028820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:00.028820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:00.028820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:00.028820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:00.028820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:00.028820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:00.028820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:00.028820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:00.028820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:00.029765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:00.029765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:00.029765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:00.029765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:00.029765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:00.029765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:00.029765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:00.029765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:00.029765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:00.029765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:00.029765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:00.029765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:00.029765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:00.029765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:00.029765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:00.029765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:00.029765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:00.030693 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:00.030693 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:00.030693 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:00.032685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 422/847] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:00.038606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c7a12-e92c-4336-ae06-9bba6efda9ea None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:00.134668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:00.142895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c7a12-e92c-4336-ae06-9bba6efda9ea None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:00.179335 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0210f99-d8f2-4896-9851-9031c2e9701a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:00.180132 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0210f99-d8f2-4896-9851-9031c2e9701a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:00.180499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0210f99-d8f2-4896-9851-9031c2e9701a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:00.180908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0210f99-d8f2-4896-9851-9031c2e9701a None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:00.181648 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0210f99-d8f2-4896-9851-9031c2e9701a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:00.182052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0210f99-d8f2-4896-9851-9031c2e9701a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:00.183533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0210f99-d8f2-4896-9851-9031c2e9701a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:00.198546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 426/848] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:00.203898 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf82e4-c072-42d8-90ca-81d52521c3f4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:00.262626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:00.309493 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf82e4-c072-42d8-90ca-81d52521c3f4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:00.348541 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b8b06-d599-49c2-85c3-64302566aae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:00.349416 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b8b06-d599-49c2-85c3-64302566aae8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:00.349841 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b8b06-d599-49c2-85c3-64302566aae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:00.350278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b8b06-d599-49c2-85c3-64302566aae8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:00.357666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b8b06-d599-49c2-85c3-64302566aae8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:00.359343 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b8b06-d599-49c2-85c3-64302566aae8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:00.371373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6a7b8b06-d599-49c2-85c3-64302566aae8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:00.371373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:00.371373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:00.371373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:00.371373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:00.371373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:00.371373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:00.371373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:00.371373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:00.371373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:00.371373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:00.371373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:00.371373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:00.371373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:00.371373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:00.372479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:00.372479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:00.372479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:00.372479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:00.372479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:00.372479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:00.372479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:00.372479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:00.372479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:00.372479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:00.372479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:00.372479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:00.372479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:00.372479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:00.372479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:00.372479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:00.372479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:00.373553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:00.373553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:00.373553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:00.373553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:00.373553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:00.373553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:00.373553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:00.373553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:00.375508 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 423/849] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:00.383388 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0210f99-d8f2-4896-9851-9031c2e9701a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:00.460259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a0297-b2d5-4e5c-a5cc-6aa40d1190e2 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:00.460546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a0297-b2d5-4e5c-a5cc-6aa40d1190e2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:00.460783 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a0297-b2d5-4e5c-a5cc-6aa40d1190e2 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:00.462396 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 424/850] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:00.467280 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fefa5-0f77-4fe4-b9a2-4cabf3975c84 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:00.467562 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fefa5-0f77-4fe4-b9a2-4cabf3975c84 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:00.467838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fefa5-0f77-4fe4-b9a2-4cabf3975c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:00.486669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-415fefa5-0f77-4fe4-b9a2-4cabf3975c84 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:00.487865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-415fefa5-0f77-4fe4-b9a2-4cabf3975c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:00.502482 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0210f99-d8f2-4896-9851-9031c2e9701a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:00.545857 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de3196-4139-41b9-88d9-cfb41d9df133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:00.546809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de3196-4139-41b9-88d9-cfb41d9df133 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:00.547089 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de3196-4139-41b9-88d9-cfb41d9df133 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:00.547367 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de3196-4139-41b9-88d9-cfb41d9df133 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:00.548323 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de3196-4139-41b9-88d9-cfb41d9df133 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:00.548658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de3196-4139-41b9-88d9-cfb41d9df133 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:00.550884 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de3196-4139-41b9-88d9-cfb41d9df133 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:00.569054 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 427/851] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:00.574453 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de3196-4139-41b9-88d9-cfb41d9df133 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:00.578324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 425/852] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:14:00.596134 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:00.599412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415fefa5-0f77-4fe4-b9a2-4cabf3975c84 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:00.660212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:00.697073 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de3196-4139-41b9-88d9-cfb41d9df133 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:00.717755 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415fefa5-0f77-4fe4-b9a2-4cabf3975c84 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:00.734295 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298d079-55eb-4100-bcff-fa711262c52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:00.735274 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298d079-55eb-4100-bcff-fa711262c52c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:00.735776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298d079-55eb-4100-bcff-fa711262c52c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:00.736249 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298d079-55eb-4100-bcff-fa711262c52c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:00.737173 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298d079-55eb-4100-bcff-fa711262c52c None admin] RBAC: Adding query filter params (role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=ac60b4d621234225830c94edfac400e2) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:00.737780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298d079-55eb-4100-bcff-fa711262c52c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:00.739608 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298d079-55eb-4100-bcff-fa711262c52c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:00.746226 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 428/853] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:00 2022] GET /identity/v3/role_assignments?user.id=ac60b4d621234225830c94edfac400e2&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:00.747582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c343a0c-9869-457c-b9c7-2eb5b17b5496 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:00.747865 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c343a0c-9869-457c-b9c7-2eb5b17b5496 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:00.748139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c343a0c-9869-457c-b9c7-2eb5b17b5496 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:00.750318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 429/854] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:00.755666 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6a7a5-db1d-402d-9698-a08aaa3dba4f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:00.756162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6a7a5-db1d-402d-9698-a08aaa3dba4f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:00.756601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6a7a5-db1d-402d-9698-a08aaa3dba4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:00.760470 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca0280-9855-4289-b3da-482aaa9b4641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:00.761253 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca0280-9855-4289-b3da-482aaa9b4641 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:00.761637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca0280-9855-4289-b3da-482aaa9b4641 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:00.762027 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca0280-9855-4289-b3da-482aaa9b4641 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:00.762763 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca0280-9855-4289-b3da-482aaa9b4641 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:00.764308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca0280-9855-4289-b3da-482aaa9b4641 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:00.776831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ebca0280-9855-4289-b3da-482aaa9b4641 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:14:00.776831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:00.776831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:14:00.776831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:14:00.776831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:14:00.776831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:14:00.776831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:14:00.776831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:00.776831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:14:00.776831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:00.776831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:00.776831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:00.776831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:00.776831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:00.776831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:00.776831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:00.777848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:00.777848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:00.777848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:00.777848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:00.777848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:00.777848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:00.777848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:00.777848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:14:00.777848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:14:00.777848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:14:00.777848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:14:00.777848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:00.777848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:00.777848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:00.777848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:00.777848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:00.777848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:00.778776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:00.778776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:00.778776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:00.778776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:00.778776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:00.778776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:00.778776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:00.778776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:00.778776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:00.778776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:00.778776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:14:00.778776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:00.778776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:14:00.778776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:00.781563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-42f6a7a5-db1d-402d-9698-a08aaa3dba4f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:00.783089 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-42f6a7a5-db1d-402d-9698-a08aaa3dba4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:00.783227 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 426/855] 10.222.0.39 () {56 vars in 1132 bytes} [Mon Aug 8 02:14:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:00.784179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca0280-9855-4289-b3da-482aaa9b4641 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:00.886399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 430/856] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:00.911083 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6a7a5-db1d-402d-9698-a08aaa3dba4f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:00.921419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca0280-9855-4289-b3da-482aaa9b4641 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:00.921779 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:00.937924 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:00.966309 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b0e6f-67a2-4f16-acf1-b0a33a28b757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:00.966855 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b0e6f-67a2-4f16-acf1-b0a33a28b757 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:00.967137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b0e6f-67a2-4f16-acf1-b0a33a28b757 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:00.967411 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b0e6f-67a2-4f16-acf1-b0a33a28b757 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:00.968036 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b0e6f-67a2-4f16-acf1-b0a33a28b757 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:00.968340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b0e6f-67a2-4f16-acf1-b0a33a28b757 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:00.969734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b0e6f-67a2-4f16-acf1-b0a33a28b757 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:00.976592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 427/857] 10.222.0.39 () {56 vars in 1126 bytes} [Mon Aug 8 02:14:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:00.982180 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b0e6f-67a2-4f16-acf1-b0a33a28b757 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:01.023726 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6a7a5-db1d-402d-9698-a08aaa3dba4f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:01.071185 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceee0c8-cf67-4c7c-bb50-941ec8b242ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:01.071812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceee0c8-cf67-4c7c-bb50-941ec8b242ad None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.072085 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceee0c8-cf67-4c7c-bb50-941ec8b242ad None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.072359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceee0c8-cf67-4c7c-bb50-941ec8b242ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.084701 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceee0c8-cf67-4c7c-bb50-941ec8b242ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:01.089495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-9ceee0c8-cf67-4c7c-bb50-941ec8b242ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:01.089495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:01.089495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:01.089495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:01.089495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:01.089495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:01.089495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:01.089495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:01.089495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:01.089495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:01.089495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:01.089495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:01.089495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:01.089495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:01.089495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:01.089495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:01.090431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:01.090431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:01.090431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:01.090431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:01.090431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:01.090431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:01.090431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:01.090431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:01.090431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:01.090431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:01.090431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:01.090431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:01.090431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:01.090431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:01.090431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:01.090431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:01.091640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:01.091640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:01.091640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:01.091640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:01.091640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:01.091640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:01.091640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:01.091640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:01.091640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:01.091640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:01.091640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:01.091640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:01.091640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 431/858] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:01.098874 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceee0c8-cf67-4c7c-bb50-941ec8b242ad None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:01.112174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b0e6f-67a2-4f16-acf1-b0a33a28b757 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:01.157179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450187a-dfe1-43c7-a802-0a21a6602a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:01.157787 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450187a-dfe1-43c7-a802-0a21a6602a31 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.158229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450187a-dfe1-43c7-a802-0a21a6602a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.158531 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450187a-dfe1-43c7-a802-0a21a6602a31 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.159539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450187a-dfe1-43c7-a802-0a21a6602a31 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:01.159985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450187a-dfe1-43c7-a802-0a21a6602a31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:01.161663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450187a-dfe1-43c7-a802-0a21a6602a31 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:01.174499 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 428/859] 10.222.0.39 () {56 vars in 1126 bytes} [Mon Aug 8 02:14:00 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:01.189770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:01.229648 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceee0c8-cf67-4c7c-bb50-941ec8b242ad None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:01.268116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97d179-ede3-4e30-99f0-9a1611ea2a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:01.268781 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97d179-ede3-4e30-99f0-9a1611ea2a53 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.269057 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97d179-ede3-4e30-99f0-9a1611ea2a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.269360 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97d179-ede3-4e30-99f0-9a1611ea2a53 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.270036 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97d179-ede3-4e30-99f0-9a1611ea2a53 None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:01.270366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97d179-ede3-4e30-99f0-9a1611ea2a53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:01.271742 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97d179-ede3-4e30-99f0-9a1611ea2a53 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:01.281534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 432/860] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:14:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:01.291423 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450187a-dfe1-43c7-a802-0a21a6602a31 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:01.337881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:01.402088 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450187a-dfe1-43c7-a802-0a21a6602a31 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:01.446724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65d095-f489-4164-83ae-08d099d83f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:01.447356 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65d095-f489-4164-83ae-08d099d83f19 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.447628 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65d095-f489-4164-83ae-08d099d83f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.447894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65d095-f489-4164-83ae-08d099d83f19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.459322 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65d095-f489-4164-83ae-08d099d83f19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:01.460770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65d095-f489-4164-83ae-08d099d83f19 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:01.475678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1c65d095-f489-4164-83ae-08d099d83f19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:01.475678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:01.475678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:01.475678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:01.475678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:01.475678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:01.475678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:01.475678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:01.475678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:01.475678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:01.475678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:01.475678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:01.475678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:01.475678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:01.475678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:01.475678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:01.478116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:01.478116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:01.478116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:01.478116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:01.478116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:01.478116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:01.478116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:01.478116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:01.478116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:01.478116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:01.478116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:01.478116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:01.478116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:01.478116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:01.478116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:01.478116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:01.478116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:01.480389 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:01.480389 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:01.480389 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:01.480389 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 429/861] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:01.482942 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97d179-ede3-4e30-99f0-9a1611ea2a53 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:01.588134 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97d179-ede3-4e30-99f0-9a1611ea2a53 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:01.618227 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:01.625016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a8a15-3f5d-4e83-8058-97ae7ec63d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:01.625816 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a8a15-3f5d-4e83-8058-97ae7ec63d91 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.626278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a8a15-3f5d-4e83-8058-97ae7ec63d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.626620 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a8a15-3f5d-4e83-8058-97ae7ec63d91 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.627278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a8a15-3f5d-4e83-8058-97ae7ec63d91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:01.627588 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a8a15-3f5d-4e83-8058-97ae7ec63d91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:01.628991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a8a15-3f5d-4e83-8058-97ae7ec63d91 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:01.641636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 433/862] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:01.650570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65d095-f489-4164-83ae-08d099d83f19 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:01.745782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:01.756789 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65d095-f489-4164-83ae-08d099d83f19 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:01.796182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abf262-3dcc-49c4-93e8-e1aaf2c895ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:01.796796 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abf262-3dcc-49c4-93e8-e1aaf2c895ff None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.797069 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abf262-3dcc-49c4-93e8-e1aaf2c895ff None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.797377 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abf262-3dcc-49c4-93e8-e1aaf2c895ff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.805573 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abf262-3dcc-49c4-93e8-e1aaf2c895ff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:01.807051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abf262-3dcc-49c4-93e8-e1aaf2c895ff None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:01.820218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a9abf262-3dcc-49c4-93e8-e1aaf2c895ff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:14:01.820218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:01.820218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:01.820218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:01.820218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:01.820218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:01.820218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:01.820218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:01.820218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:01.820218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:01.820218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:01.820218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:01.820218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:01.820218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:01.820218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:01.820218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:01.822581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:01.822581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:01.822581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:01.822581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:01.822581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:01.822581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:01.822581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:01.822581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:01.822581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:01.822581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:01.822581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:01.822581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:01.822581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:01.822581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:01.822581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:01.822581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:01.822581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:01.822581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:01.824996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:01.824996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:01.824996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:01.824996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:01.824996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:01.824996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:01.824996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:01.824996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:01.824996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:14:01.824996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:01.826665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 430/863] 10.222.0.39 () {56 vars in 1117 bytes} [Mon Aug 8 02:14:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:01.827335 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a8a15-3f5d-4e83-8058-97ae7ec63d91 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:01.939120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a8a15-3f5d-4e83-8058-97ae7ec63d91 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:01.968462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890e971-5219-4774-8257-281c2ccebac4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.968781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890e971-5219-4774-8257-281c2ccebac4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.969020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890e971-5219-4774-8257-281c2ccebac4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.970659 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 431/864] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:14:01.976679 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561d6a9-ff65-4f03-ad9c-202f466f10a7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.977029 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561d6a9-ff65-4f03-ad9c-202f466f10a7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.977243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561d6a9-ff65-4f03-ad9c-202f466f10a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.980676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483784c6-49bf-413e-9585-747c8580dde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:01.981514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483784c6-49bf-413e-9585-747c8580dde9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.981967 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483784c6-49bf-413e-9585-747c8580dde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.982410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483784c6-49bf-413e-9585-747c8580dde9 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.983308 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483784c6-49bf-413e-9585-747c8580dde9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:01.984001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483784c6-49bf-413e-9585-747c8580dde9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:01.986692 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483784c6-49bf-413e-9585-747c8580dde9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:01.997111 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3561d6a9-ff65-4f03-ad9c-202f466f10a7 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:01.998716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3561d6a9-ff65-4f03-ad9c-202f466f10a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:02.008766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 434/865] 10.222.0.39 () {56 vars in 1149 bytes} [Mon Aug 8 02:14:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:02.017159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483784c6-49bf-413e-9585-747c8580dde9 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:02.076557 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:02.094290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 432/866] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:02.112193 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561d6a9-ff65-4f03-ad9c-202f466f10a7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:02.146467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483784c6-49bf-413e-9585-747c8580dde9 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:02.188466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c954c7e-99c8-401f-9592-6378bfbce696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:02.189080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c954c7e-99c8-401f-9592-6378bfbce696 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:02.189360 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c954c7e-99c8-401f-9592-6378bfbce696 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:02.189636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c954c7e-99c8-401f-9592-6378bfbce696 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:02.200868 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:02.223902 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561d6a9-ff65-4f03-ad9c-202f466f10a7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:02.261261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96294698-ae7d-4232-b69a-b0126bcb0ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:02.262211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96294698-ae7d-4232-b69a-b0126bcb0ef4 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:02.262738 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96294698-ae7d-4232-b69a-b0126bcb0ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:02.263269 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96294698-ae7d-4232-b69a-b0126bcb0ef4 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:02.264221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96294698-ae7d-4232-b69a-b0126bcb0ef4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:02.265846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96294698-ae7d-4232-b69a-b0126bcb0ef4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:02.271160 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c954c7e-99c8-401f-9592-6378bfbce696 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:02.273165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c954c7e-99c8-401f-9592-6378bfbce696 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:02.283720 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 433/867] 10.222.0.39 () {60 vars in 1141 bytes} [Mon Aug 8 02:14:02 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:14:02.344282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:02.408401 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 435/868] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:02 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:14:02.414598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96294698-ae7d-4232-b69a-b0126bcb0ef4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:02.536143 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96294698-ae7d-4232-b69a-b0126bcb0ef4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:02.571868 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa644a-8ebf-495c-a2d8-314298f8e0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:02.572930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa644a-8ebf-495c-a2d8-314298f8e0dc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:02.573418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa644a-8ebf-495c-a2d8-314298f8e0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:02.573943 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa644a-8ebf-495c-a2d8-314298f8e0dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:02.577102 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:02.588498 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa644a-8ebf-495c-a2d8-314298f8e0dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:02.590020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa644a-8ebf-495c-a2d8-314298f8e0dc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:02.605654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ddaa644a-8ebf-495c-a2d8-314298f8e0dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:02.605654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:02.605654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:02.605654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:02.605654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:02.605654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:02.605654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:02.605654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:02.605654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:02.605654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:02.605654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:02.605654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:02.605654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:02.605654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:02.605654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:02.605654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:02.606649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:02.606649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:02.606649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:02.606649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:02.606649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:02.606649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:02.606649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:02.606649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:02.606649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:02.606649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:02.606649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:02.606649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:02.606649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:02.606649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:02.606649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:02.606649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:02.606649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:02.607507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:02.607507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:02.607507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:02.607507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 434/869] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:02.612715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c954c7e-99c8-401f-9592-6378bfbce696 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:02.716291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c954c7e-99c8-401f-9592-6378bfbce696 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:02.751995 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201835b-d8a8-42db-ae70-610ffd6d0845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:02.752596 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201835b-d8a8-42db-ae70-610ffd6d0845 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:02.752870 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201835b-d8a8-42db-ae70-610ffd6d0845 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:02.753138 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201835b-d8a8-42db-ae70-610ffd6d0845 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:02.753820 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201835b-d8a8-42db-ae70-610ffd6d0845 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:02.754120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201835b-d8a8-42db-ae70-610ffd6d0845 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:02.755496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201835b-d8a8-42db-ae70-610ffd6d0845 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:02.767438 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 436/870] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:02.773398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa644a-8ebf-495c-a2d8-314298f8e0dc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:02.888221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa644a-8ebf-495c-a2d8-314298f8e0dc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:02.901790 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45766d7-1574-4669-be28-bd285c269141 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:02.902083 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45766d7-1574-4669-be28-bd285c269141 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:02.902395 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45766d7-1574-4669-be28-bd285c269141 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:02.903831 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 437/871] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:14:02.909426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7845a83-f296-4eaf-a441-70aa6b0591cc None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:02.909708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7845a83-f296-4eaf-a441-70aa6b0591cc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:02.910022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7845a83-f296-4eaf-a441-70aa6b0591cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:02.928103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c7845a83-f296-4eaf-a441-70aa6b0591cc None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:02.928496 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd651a-f2e7-43b6-9fc4-c2976e4b4ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:02.929291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c7845a83-f296-4eaf-a441-70aa6b0591cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:02.929600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd651a-f2e7-43b6-9fc4-c2976e4b4ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:02.930076 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd651a-f2e7-43b6-9fc4-c2976e4b4ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:02.930618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd651a-f2e7-43b6-9fc4-c2976e4b4ca2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:02.931307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:02.938333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd651a-f2e7-43b6-9fc4-c2976e4b4ca2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:02.939762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd651a-f2e7-43b6-9fc4-c2976e4b4ca2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:02.952131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-31bd651a-f2e7-43b6-9fc4-c2976e4b4ca2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:02.952131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:02.952131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:02.952131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:02.952131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:02.952131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:02.952131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:02.952131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:02.952131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:02.952131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:02.952131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:02.952131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:02.952131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:02.952131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:02.952131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:02.953074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:02.953074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:02.953074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:02.953074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:02.953074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:02.953074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:02.953074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:02.953074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:02.953074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:02.953074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:02.953074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:02.953074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:02.953074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:02.953074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:02.953074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:02.953074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:02.953074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:02.953962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:02.953962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:02.953962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:02.953962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:02.953962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:02.953962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:02.953962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:02.953962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:02.953962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 435/872] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:02.959677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd651a-f2e7-43b6-9fc4-c2976e4b4ca2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:03.012926 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:03.014393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 438/873] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:02 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:03.032318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7845a83-f296-4eaf-a441-70aa6b0591cc None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:03.071033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd651a-f2e7-43b6-9fc4-c2976e4b4ca2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:03.108319 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f543ecc-3798-40eb-83b6-94c8b244e596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:03.109308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f543ecc-3798-40eb-83b6-94c8b244e596 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:03.109824 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f543ecc-3798-40eb-83b6-94c8b244e596 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:03.110282 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f543ecc-3798-40eb-83b6-94c8b244e596 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:03.111157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f543ecc-3798-40eb-83b6-94c8b244e596 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:03.111645 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f543ecc-3798-40eb-83b6-94c8b244e596 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:03.113314 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f543ecc-3798-40eb-83b6-94c8b244e596 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:03.126811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 436/874] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:03.133613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f543ecc-3798-40eb-83b6-94c8b244e596 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:03.141430 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7845a83-f296-4eaf-a441-70aa6b0591cc None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:03.178885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3cf60-82ff-497a-a8de-82abe5d5fa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:03.179843 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3cf60-82ff-497a-a8de-82abe5d5fa71 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:03.179970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3cf60-82ff-497a-a8de-82abe5d5fa71 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:03.180310 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3cf60-82ff-497a-a8de-82abe5d5fa71 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:03.180958 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa3cf60-82ff-497a-a8de-82abe5d5fa71 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:03.182562 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa3cf60-82ff-497a-a8de-82abe5d5fa71 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:03.192885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-9aa3cf60-82ff-497a-a8de-82abe5d5fa71 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:14:03.192885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:03.192885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:14:03.192885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:14:03.192885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:14:03.192885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:14:03.192885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:14:03.192885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:03.192885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:14:03.192885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:03.192885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:03.192885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:03.192885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:03.192885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:03.192885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:03.192885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:03.192885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:03.193882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:03.193882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:03.193882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:03.193882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:03.193882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:03.193882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:03.193882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:14:03.193882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:14:03.193882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:14:03.193882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:14:03.193882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:03.193882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:03.193882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:03.193882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:03.193882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:03.193882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:03.194980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:03.194980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:03.194980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:03.194980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:03.194980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:03.194980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:03.194980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:03.194980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:03.194980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:03.194980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:03.194980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:14:03.194980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:03.194980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:14:03.194980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:03.199370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 439/875] 10.222.0.39 () {56 vars in 1126 bytes} [Mon Aug 8 02:14:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:03.200271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3cf60-82ff-497a-a8de-82abe5d5fa71 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:03.203300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:03.247388 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f543ecc-3798-40eb-83b6-94c8b244e596 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:03.258467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:03.285430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9cb7ef-1c66-4254-8993-2be16d85a6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:03.286032 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cb7ef-1c66-4254-8993-2be16d85a6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:03.286338 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cb7ef-1c66-4254-8993-2be16d85a6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:03.286610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cb7ef-1c66-4254-8993-2be16d85a6e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:03.287457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9cb7ef-1c66-4254-8993-2be16d85a6e9 None admin] RBAC: Adding query filter params (role.id=838b081e674f4c0ab4a33bcf8e614dc5, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=b88ccd3e5c4c48d3a27f4eaa7874e28e) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:03.287772 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9cb7ef-1c66-4254-8993-2be16d85a6e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:03.289339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9cb7ef-1c66-4254-8993-2be16d85a6e9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:03.302590 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 437/876] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:03 2022] GET /identity/v3/role_assignments?user.id=b88ccd3e5c4c48d3a27f4eaa7874e28e&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=838b081e674f4c0ab4a33bcf8e614dc5 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:03.303866 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d705379-4422-4b6f-a052-fe54268680e3 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:03.304156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d705379-4422-4b6f-a052-fe54268680e3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:03.304419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d705379-4422-4b6f-a052-fe54268680e3 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:03.305915 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 438/877] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:03.311732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2123dd3-9c38-4098-9ff9-45d6c1e1af3c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:03.312031 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2123dd3-9c38-4098-9ff9-45d6c1e1af3c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:03.312281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2123dd3-9c38-4098-9ff9-45d6c1e1af3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:03.323288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3cf60-82ff-497a-a8de-82abe5d5fa71 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:03.332122 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c2123dd3-9c38-4098-9ff9-45d6c1e1af3c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:03.333439 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c2123dd3-9c38-4098-9ff9-45d6c1e1af3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:03.366615 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad4ff8-67b0-4672-aa3a-938768b7a39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:03.367894 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad4ff8-67b0-4672-aa3a-938768b7a39d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:03.368857 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad4ff8-67b0-4672-aa3a-938768b7a39d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:03.369612 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad4ff8-67b0-4672-aa3a-938768b7a39d None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:03.370862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad4ff8-67b0-4672-aa3a-938768b7a39d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:03.371493 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad4ff8-67b0-4672-aa3a-938768b7a39d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:03.373098 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad4ff8-67b0-4672-aa3a-938768b7a39d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:03.385659 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 440/878] 10.222.0.39 () {56 vars in 1122 bytes} [Mon Aug 8 02:14:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:03.389198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad4ff8-67b0-4672-aa3a-938768b7a39d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:03.423449 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 439/879] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:03 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:14:03.447382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2123dd3-9c38-4098-9ff9-45d6c1e1af3c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:03.494760 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:03.544159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad4ff8-67b0-4672-aa3a-938768b7a39d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:03.586512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e0acdf-b7ce-4e91-9d97-a6f90ac3c6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:03.587216 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0acdf-b7ce-4e91-9d97-a6f90ac3c6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:03.587369 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2123dd3-9c38-4098-9ff9-45d6c1e1af3c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:03.587625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0acdf-b7ce-4e91-9d97-a6f90ac3c6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:03.587966 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0acdf-b7ce-4e91-9d97-a6f90ac3c6b6 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:03.588705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0acdf-b7ce-4e91-9d97-a6f90ac3c6b6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:03.589018 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0acdf-b7ce-4e91-9d97-a6f90ac3c6b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:03.589150 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:03.590541 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0acdf-b7ce-4e91-9d97-a6f90ac3c6b6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:03.595509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 441/880] 10.222.0.39 () {56 vars in 1122 bytes} [Mon Aug 8 02:14:03 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:03.628007 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a98ac2-acdd-497d-9ef0-2bf440abff17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:03.628656 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a98ac2-acdd-497d-9ef0-2bf440abff17 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:03.628943 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a98ac2-acdd-497d-9ef0-2bf440abff17 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:03.629215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a98ac2-acdd-497d-9ef0-2bf440abff17 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:03.629880 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a98ac2-acdd-497d-9ef0-2bf440abff17 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:03.631284 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a98ac2-acdd-497d-9ef0-2bf440abff17 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:03.650078 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 440/881] 10.222.0.39 () {60 vars in 1141 bytes} [Mon Aug 8 02:14:03 2022] POST /identity/v3/services => generated 269 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:14:04.458223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3fe5e-0124-4dee-859d-39808d64712d None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:04.458747 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3fe5e-0124-4dee-859d-39808d64712d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:04.459026 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3fe5e-0124-4dee-859d-39808d64712d None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:04.460707 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 442/882] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:04.465889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3af683-8e2f-45f4-88f9-4638f2a2a9a8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:04.466299 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3af683-8e2f-45f4-88f9-4638f2a2a9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:04.466641 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3af683-8e2f-45f4-88f9-4638f2a2a9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:04.485148 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1d3af683-8e2f-45f4-88f9-4638f2a2a9a8 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:04.486365 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1d3af683-8e2f-45f4-88f9-4638f2a2a9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:04.558699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 441/883] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:04.581588 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3fe5e-0124-4dee-859d-39808d64712d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:04.687758 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3fe5e-0124-4dee-859d-39808d64712d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:04.716755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:04.723963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767583b4-9412-4ddd-9b59-eb484f472bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:04.724905 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767583b4-9412-4ddd-9b59-eb484f472bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:04.725545 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767583b4-9412-4ddd-9b59-eb484f472bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:04.725824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767583b4-9412-4ddd-9b59-eb484f472bc9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:04.726446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767583b4-9412-4ddd-9b59-eb484f472bc9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:04.727814 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767583b4-9412-4ddd-9b59-eb484f472bc9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:04.736625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-767583b4-9412-4ddd-9b59-eb484f472bc9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:14:04.736625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:04.736625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:14:04.736625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:14:04.736625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:14:04.736625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:14:04.736625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:14:04.736625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:04.736625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:14:04.736625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:04.736625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:04.736625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:04.736625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:04.736625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:04.736625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:04.736625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:04.736625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:04.737564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:04.737564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:04.737564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:04.737564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:04.737564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:04.737564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:04.737564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:14:04.737564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:14:04.737564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:14:04.737564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:14:04.737564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:04.737564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:04.737564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:04.737564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:04.737564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:04.737564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:04.738433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:04.738433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:04.738433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:04.738433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:04.738433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:04.738433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:04.738433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:04.738433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:04.738433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:04.738433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:04.738433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:14:04.738433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:04.738433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:14:04.738433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:04.738433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 443/884] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:14:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:04.743940 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3af683-8e2f-45f4-88f9-4638f2a2a9a8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:04.836798 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:04.851351 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3af683-8e2f-45f4-88f9-4638f2a2a9a8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:04.885905 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c03615-1c55-4f43-9eb7-d7ef5fc6ff21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:04.886831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c03615-1c55-4f43-9eb7-d7ef5fc6ff21 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:04.887250 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c03615-1c55-4f43-9eb7-d7ef5fc6ff21 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:04.887680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c03615-1c55-4f43-9eb7-d7ef5fc6ff21 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:04.888442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c03615-1c55-4f43-9eb7-d7ef5fc6ff21 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:04.888874 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c03615-1c55-4f43-9eb7-d7ef5fc6ff21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:04.890412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c03615-1c55-4f43-9eb7-d7ef5fc6ff21 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:04.896060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 442/885] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:14:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:04.901454 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767583b4-9412-4ddd-9b59-eb484f472bc9 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:05.004715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767583b4-9412-4ddd-9b59-eb484f472bc9 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:05.039613 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2c498-9261-4263-abc9-b5798de0839c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:05.040216 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2c498-9261-4263-abc9-b5798de0839c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.040489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2c498-9261-4263-abc9-b5798de0839c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.040756 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2c498-9261-4263-abc9-b5798de0839c None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.041378 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b2c498-9261-4263-abc9-b5798de0839c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:05.041678 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b2c498-9261-4263-abc9-b5798de0839c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:05.043045 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b2c498-9261-4263-abc9-b5798de0839c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:05.048289 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 444/886] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:14:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:05.055347 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c03615-1c55-4f43-9eb7-d7ef5fc6ff21 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:05.159124 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c03615-1c55-4f43-9eb7-d7ef5fc6ff21 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:05.194465 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c8252-eac8-4606-9b63-2488fdd4421c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:05.195072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c8252-eac8-4606-9b63-2488fdd4421c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.195347 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c8252-eac8-4606-9b63-2488fdd4421c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.195615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c8252-eac8-4606-9b63-2488fdd4421c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.196312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c8252-eac8-4606-9b63-2488fdd4421c None admin] RBAC: Adding query filter params (interface=public, service_id=13d74570bce543de9da625390c061190, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:05.196619 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c8252-eac8-4606-9b63-2488fdd4421c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:05.197986 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c8252-eac8-4606-9b63-2488fdd4421c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:05.205970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 443/887] 10.222.0.39 () {56 vars in 1251 bytes} [Mon Aug 8 02:14:05 2022] GET /identity/v3/endpoints?service_id=13d74570bce543de9da625390c061190&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:05.212283 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2c498-9261-4263-abc9-b5798de0839c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:05.277334 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:05.319719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2c498-9261-4263-abc9-b5798de0839c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:05.355368 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20d40f-9989-45a6-bd53-b284aaaffe32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:05.355978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20d40f-9989-45a6-bd53-b284aaaffe32 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.356253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20d40f-9989-45a6-bd53-b284aaaffe32 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.356557 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20d40f-9989-45a6-bd53-b284aaaffe32 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.357120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20d40f-9989-45a6-bd53-b284aaaffe32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:05.358490 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20d40f-9989-45a6-bd53-b284aaaffe32 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:05.365402 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 445/888] 10.222.0.39 () {56 vars in 1087 bytes} [Mon Aug 8 02:14:05 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:14:05.547132 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fef38d-04a4-47d5-89fe-806bb036cd91 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.547610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fef38d-04a4-47d5-89fe-806bb036cd91 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.547610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fef38d-04a4-47d5-89fe-806bb036cd91 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.549157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 444/889] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:05.554425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94b354-df32-4bfa-a32e-6f63583078de None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.554882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94b354-df32-4bfa-a32e-6f63583078de None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.554960 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94b354-df32-4bfa-a32e-6f63583078de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.578755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6b94b354-df32-4bfa-a32e-6f63583078de None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:05.579939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6b94b354-df32-4bfa-a32e-6f63583078de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:05.651194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 446/890] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:05.674385 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fef38d-04a4-47d5-89fe-806bb036cd91 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:05.693724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:05.705674 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19de0fd-7ada-4355-b504-0548d4b28afc None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.706003 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19de0fd-7ada-4355-b504-0548d4b28afc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.706214 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19de0fd-7ada-4355-b504-0548d4b28afc None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.708074 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 447/891] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:14:05.714028 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b7560-f4ce-4741-8d80-48b5ea295f97 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.714348 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b7560-f4ce-4741-8d80-48b5ea295f97 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.714628 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b7560-f4ce-4741-8d80-48b5ea295f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.734951 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-988b7560-f4ce-4741-8d80-48b5ea295f97 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:05.736231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-988b7560-f4ce-4741-8d80-48b5ea295f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:05.791179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:05.797467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fef38d-04a4-47d5-89fe-806bb036cd91 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:05.815298 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 448/892] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:14:05.818553 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6d666-2489-48b0-990b-218d9d94b0ed None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.818667 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6d666-2489-48b0-990b-218d9d94b0ed None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.818866 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6d666-2489-48b0-990b-218d9d94b0ed None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.820228 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 449/893] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:05.828908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505b1ad-96e4-475d-9c90-f4d37a1ebbd1 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.829439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505b1ad-96e4-475d-9c90-f4d37a1ebbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.829439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505b1ad-96e4-475d-9c90-f4d37a1ebbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.846134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b63508-a89c-4d8f-a9ad-783838f130ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:05.846795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b63508-a89c-4d8f-a9ad-783838f130ff None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.847070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b63508-a89c-4d8f-a9ad-783838f130ff None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.847873 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b63508-a89c-4d8f-a9ad-783838f130ff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.848449 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8505b1ad-96e4-475d-9c90-f4d37a1ebbd1 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:05.849634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8505b1ad-96e4-475d-9c90-f4d37a1ebbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:05.858719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b63508-a89c-4d8f-a9ad-783838f130ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:05.864123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-14b63508-a89c-4d8f-a9ad-783838f130ff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:14:05.864123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:05.864123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:05.864123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:05.864123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:05.864123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:05.864123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:05.864123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:05.864123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:05.864123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:05.864123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:05.864123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:05.864123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:05.864123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:05.864123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:05.864123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:05.865023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:05.865023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:05.865023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:05.865023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:05.865023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:05.865023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:05.865023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:05.865023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:05.865023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:05.865023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:05.865023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:05.865023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:05.865023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:05.865023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:05.865023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:05.865023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:05.866802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:05.866802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:05.866802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:05.866802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:05.866802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:05.866802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:05.866802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:05.866802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:05.866802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:05.866802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:05.866802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:14:05.866802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:05.866802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 445/894] 10.222.0.39 () {56 vars in 1120 bytes} [Mon Aug 8 02:14:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:05.871090 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b63508-a89c-4d8f-a9ad-783838f130ff None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:05.926993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 450/895] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:14:05.927902 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505b1ad-96e4-475d-9c90-f4d37a1ebbd1 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:05.950837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:05.975875 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b63508-a89c-4d8f-a9ad-783838f130ff None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.011197 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5dd453-ec85-4c9a-acc2-3ac97b10f2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.011802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5dd453-ec85-4c9a-acc2-3ac97b10f2a4 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.012088 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5dd453-ec85-4c9a-acc2-3ac97b10f2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.012358 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5dd453-ec85-4c9a-acc2-3ac97b10f2a4 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:06.012998 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5dd453-ec85-4c9a-acc2-3ac97b10f2a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:06.014482 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5dd453-ec85-4c9a-acc2-3ac97b10f2a4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:06.029517 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 446/896] 10.222.0.39 () {60 vars in 1141 bytes} [Mon Aug 8 02:14:05 2022] POST /identity/v3/services => generated 241 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:14:06.030763 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5dd453-ec85-4c9a-acc2-3ac97b10f2a4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:06.057805 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505b1ad-96e4-475d-9c90-f4d37a1ebbd1 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.079016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:06.121711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9ff95-b02a-4c8d-92fe-c35add24fdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.122373 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9ff95-b02a-4c8d-92fe-c35add24fdc9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.122653 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9ff95-b02a-4c8d-92fe-c35add24fdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.122929 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9ff95-b02a-4c8d-92fe-c35add24fdc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:06.123614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9ff95-b02a-4c8d-92fe-c35add24fdc9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:06.123917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9ff95-b02a-4c8d-92fe-c35add24fdc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:06.126495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9ff95-b02a-4c8d-92fe-c35add24fdc9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:06.142667 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 451/897] 10.222.0.39 () {56 vars in 1115 bytes} [Mon Aug 8 02:14:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:06.144065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9ff95-b02a-4c8d-92fe-c35add24fdc9 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:06.162148 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5dd453-ec85-4c9a-acc2-3ac97b10f2a4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.200257 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7831436-ffef-4b6f-a2ed-5348c9ec82b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.200934 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7831436-ffef-4b6f-a2ed-5348c9ec82b8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.201209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7831436-ffef-4b6f-a2ed-5348c9ec82b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.201497 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7831436-ffef-4b6f-a2ed-5348c9ec82b8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:06.202081 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7831436-ffef-4b6f-a2ed-5348c9ec82b8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:06.203508 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7831436-ffef-4b6f-a2ed-5348c9ec82b8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:06.206538 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:06.214572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d7831436-ffef-4b6f-a2ed-5348c9ec82b8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:14:06.214572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:06.214572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:14:06.214572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:14:06.214572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:14:06.214572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:14:06.214572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:14:06.214572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:06.214572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:14:06.214572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:06.214572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:06.214572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:06.214572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:06.214572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:06.214572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:06.214572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:06.215500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:06.215500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:06.215500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:06.215500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:06.215500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:06.215500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:06.215500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:06.215500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:14:06.215500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:14:06.215500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:14:06.215500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:14:06.215500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:06.215500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:06.215500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:06.215500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:06.215500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:06.215500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:06.216428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:06.216428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:06.216428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:06.216428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:06.216428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:06.216428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:06.216428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:06.216428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:06.216428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:06.216428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:06.216428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:14:06.216428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:06.216428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:14:06.216428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:06.216428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 447/898] 10.222.0.39 () {56 vars in 1132 bytes} [Mon Aug 8 02:14:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:06.224133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7831436-ffef-4b6f-a2ed-5348c9ec82b8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:06.257801 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9ff95-b02a-4c8d-92fe-c35add24fdc9 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.292512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add63378-e666-408d-bea7-601109690f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.293305 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add63378-e666-408d-bea7-601109690f69 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.293679 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add63378-e666-408d-bea7-601109690f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.294058 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add63378-e666-408d-bea7-601109690f69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:06.310038 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add63378-e666-408d-bea7-601109690f69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:06.312293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add63378-e666-408d-bea7-601109690f69 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:06.332709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-add63378-e666-408d-bea7-601109690f69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:06.332709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:06.332709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:06.332709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:06.332709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:06.332709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:06.332709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:06.332709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:06.332709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:06.332709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:06.332709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:06.332709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:06.332709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:06.332709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:06.332709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:06.332709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:06.333877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:06.333877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:06.333877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:06.333877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:06.333877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:06.333877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:06.333877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:06.333877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:06.333877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:06.333877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:06.333877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:06.333877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:06.333877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:06.333877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:06.333877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:06.333877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:06.333877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:06.335086 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:06.335086 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:06.335086 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:06.337684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 452/899] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:06.342822 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add63378-e666-408d-bea7-601109690f69 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:06.364618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7831436-ffef-4b6f-a2ed-5348c9ec82b8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.368193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:06.411566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4a8c7-82b1-4845-a748-e827093210d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.413134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4a8c7-82b1-4845-a748-e827093210d4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.414020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4a8c7-82b1-4845-a748-e827093210d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.414516 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4a8c7-82b1-4845-a748-e827093210d4 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:06.415597 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4a8c7-82b1-4845-a748-e827093210d4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:06.416976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4a8c7-82b1-4845-a748-e827093210d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:06.418612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4a8c7-82b1-4845-a748-e827093210d4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:06.428594 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 448/900] 10.222.0.39 () {56 vars in 1126 bytes} [Mon Aug 8 02:14:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:06.434017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4a8c7-82b1-4845-a748-e827093210d4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:06.456164 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add63378-e666-408d-bea7-601109690f69 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.496615 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0337b-ceba-4af3-a472-3399a9e3a3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.497750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0337b-ceba-4af3-a472-3399a9e3a3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.498687 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0337b-ceba-4af3-a472-3399a9e3a3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.498991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0337b-ceba-4af3-a472-3399a9e3a3d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:06.499689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0337b-ceba-4af3-a472-3399a9e3a3d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:06.499998 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0337b-ceba-4af3-a472-3399a9e3a3d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:06.501456 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0337b-ceba-4af3-a472-3399a9e3a3d9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:06.520548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 453/901] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:06.521479 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0337b-ceba-4af3-a472-3399a9e3a3d9 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:06.523757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:06.571275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4a8c7-82b1-4845-a748-e827093210d4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.617423 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befb6bf-caa5-4b94-888a-95f7840f5aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.618371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befb6bf-caa5-4b94-888a-95f7840f5aef None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.618650 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befb6bf-caa5-4b94-888a-95f7840f5aef None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.618916 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befb6bf-caa5-4b94-888a-95f7840f5aef None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:06.619794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befb6bf-caa5-4b94-888a-95f7840f5aef None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:06.620100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befb6bf-caa5-4b94-888a-95f7840f5aef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:06.620780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:06.623182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befb6bf-caa5-4b94-888a-95f7840f5aef None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:06.630444 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 449/902] 10.222.0.39 () {56 vars in 1126 bytes} [Mon Aug 8 02:14:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:06.635780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befb6bf-caa5-4b94-888a-95f7840f5aef None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:06.646808 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0337b-ceba-4af3-a472-3399a9e3a3d9 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.683310 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12ef85-e7f0-4711-8dc1-f5f76f5ef96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.683935 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12ef85-e7f0-4711-8dc1-f5f76f5ef96a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.684212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12ef85-e7f0-4711-8dc1-f5f76f5ef96a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.684482 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12ef85-e7f0-4711-8dc1-f5f76f5ef96a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:06.692068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12ef85-e7f0-4711-8dc1-f5f76f5ef96a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:06.693606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12ef85-e7f0-4711-8dc1-f5f76f5ef96a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:06.705312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-8d12ef85-e7f0-4711-8dc1-f5f76f5ef96a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:14:06.705312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:06.705312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:06.705312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:06.705312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:06.705312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:06.705312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:06.705312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:06.705312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:06.705312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:06.705312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:06.705312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:06.705312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:06.705312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:06.705312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:06.705312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:06.706288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:06.706288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:06.706288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:06.706288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:06.706288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:06.706288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:06.706288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:06.706288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:06.706288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:06.706288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:06.706288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:06.706288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:06.706288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:06.706288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:06.706288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:06.706288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:06.706288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:06.706288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:06.707743 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:06.707743 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:06.707743 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:06.707743 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:06.707743 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:06.707743 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:06.707743 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:06.707743 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:06.707743 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:14:06.707743 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:06.708692 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 454/903] 10.222.0.39 () {56 vars in 1117 bytes} [Mon Aug 8 02:14:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:06.713359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12ef85-e7f0-4711-8dc1-f5f76f5ef96a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:06.762559 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befb6bf-caa5-4b94-888a-95f7840f5aef None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.798233 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:06.808853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c49d5e-24f3-4b3d-8aed-e30115f7fe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.809860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c49d5e-24f3-4b3d-8aed-e30115f7fe75 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.810146 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c49d5e-24f3-4b3d-8aed-e30115f7fe75 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.810436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c49d5e-24f3-4b3d-8aed-e30115f7fe75 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:06.811635 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c49d5e-24f3-4b3d-8aed-e30115f7fe75 None admin] RBAC: Adding query filter params (interface=public, service_id=112b430067384e149dfa843ef1b482da, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:06.811952 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c49d5e-24f3-4b3d-8aed-e30115f7fe75 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:06.813672 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c49d5e-24f3-4b3d-8aed-e30115f7fe75 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:06.824663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 450/904] 10.222.0.39 () {56 vars in 1251 bytes} [Mon Aug 8 02:14:06 2022] GET /identity/v3/endpoints?service_id=112b430067384e149dfa843ef1b482da&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:06.826763 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c49d5e-24f3-4b3d-8aed-e30115f7fe75 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:06.856162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12ef85-e7f0-4711-8dc1-f5f76f5ef96a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.901667 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace18e0c-2919-49e5-a04e-c6075c134ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.902669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace18e0c-2919-49e5-a04e-c6075c134ccd None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.903169 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace18e0c-2919-49e5-a04e-c6075c134ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.903670 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace18e0c-2919-49e5-a04e-c6075c134ccd None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:06.904589 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace18e0c-2919-49e5-a04e-c6075c134ccd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:06.905104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace18e0c-2919-49e5-a04e-c6075c134ccd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:06.906784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace18e0c-2919-49e5-a04e-c6075c134ccd None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:06.923330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:06.928385 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 455/905] 10.222.0.39 () {56 vars in 1149 bytes} [Mon Aug 8 02:14:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:06.933664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace18e0c-2919-49e5-a04e-c6075c134ccd None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:06.952140 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c49d5e-24f3-4b3d-8aed-e30115f7fe75 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.991682 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d6683-6516-4f88-80dd-8752de555b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.992679 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d6683-6516-4f88-80dd-8752de555b75 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.993170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d6683-6516-4f88-80dd-8752de555b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.993773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d6683-6516-4f88-80dd-8752de555b75 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:06.994719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d6683-6516-4f88-80dd-8752de555b75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:06.996297 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d6683-6516-4f88-80dd-8752de555b75 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:07.010595 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 451/906] 10.222.0.39 () {56 vars in 1087 bytes} [Mon Aug 8 02:14:06 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:14:07.056497 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace18e0c-2919-49e5-a04e-c6075c134ccd None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:07.095928 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a164ff-ca50-4b7b-a329-ce125837f2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:07.096462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a164ff-ca50-4b7b-a329-ce125837f2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:07.096737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a164ff-ca50-4b7b-a329-ce125837f2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:07.097005 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a164ff-ca50-4b7b-a329-ce125837f2c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:07.170452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:07.173862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a164ff-ca50-4b7b-a329-ce125837f2c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:07.175209 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a164ff-ca50-4b7b-a329-ce125837f2c7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:07.278389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 456/907] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:06 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 02:14:07.283652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d6683-6516-4f88-80dd-8752de555b75 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:07.345557 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:07.391965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d6683-6516-4f88-80dd-8752de555b75 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:07.428266 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd76c1f-0396-4357-b294-ed491bc87d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:07.429182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd76c1f-0396-4357-b294-ed491bc87d11 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:07.429656 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd76c1f-0396-4357-b294-ed491bc87d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:07.430099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd76c1f-0396-4357-b294-ed491bc87d11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:07.436771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bd6a3-fbaf-4306-a80a-2efe14c6c791 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:07.437050 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bd6a3-fbaf-4306-a80a-2efe14c6c791 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:07.437317 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bd6a3-fbaf-4306-a80a-2efe14c6c791 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:07.439079 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 457/908] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:14:07.444475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1777358-df54-4bb4-8d22-bbc13872096c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:07.444754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1777358-df54-4bb4-8d22-bbc13872096c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:07.445167 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1777358-df54-4bb4-8d22-bbc13872096c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:07.445376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd76c1f-0396-4357-b294-ed491bc87d11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:07.447068 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd76c1f-0396-4357-b294-ed491bc87d11 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:07.463978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f1777358-df54-4bb4-8d22-bbc13872096c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:07.465127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-9bd76c1f-0396-4357-b294-ed491bc87d11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:07.465127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:07.465127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:07.465127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:07.465127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:07.465127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:07.465127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:07.465127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:07.465127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:07.465127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:07.465127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:07.465127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:07.465127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:07.465127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:07.465127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:07.465127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:07.466042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:07.466042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:07.466042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:07.466042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:07.466042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:07.466042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:07.466042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:07.466042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:07.466042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:07.466042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:07.466042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:07.466042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:07.466042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:07.466042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:07.466042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:07.466042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:07.466042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:07.466949 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:07.466949 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:07.466949 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:07.467125 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f1777358-df54-4bb4-8d22-bbc13872096c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:07.469067 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 452/909] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:07.474554 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd76c1f-0396-4357-b294-ed491bc87d11 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:07.550504 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 458/910] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:14:07.569831 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1777358-df54-4bb4-8d22-bbc13872096c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:07.585099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd76c1f-0396-4357-b294-ed491bc87d11 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:07.595734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:07.607842 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:07.624409 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79713c-ce05-48be-8d9c-7bed7598b777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:07.625168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79713c-ce05-48be-8d9c-7bed7598b777 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:07.625544 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79713c-ce05-48be-8d9c-7bed7598b777 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:07.625913 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79713c-ce05-48be-8d9c-7bed7598b777 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:07.626686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79713c-ce05-48be-8d9c-7bed7598b777 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:07.627104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79713c-ce05-48be-8d9c-7bed7598b777 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:07.629099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79713c-ce05-48be-8d9c-7bed7598b777 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:07.641600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 453/911] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:07.651883 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79713c-ce05-48be-8d9c-7bed7598b777 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:07.687664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1777358-df54-4bb4-8d22-bbc13872096c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:07.725452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84ef3e-4653-4960-b3db-11cf4dd4ffe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:07.726351 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84ef3e-4653-4960-b3db-11cf4dd4ffe6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:07.726770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84ef3e-4653-4960-b3db-11cf4dd4ffe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:07.727276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84ef3e-4653-4960-b3db-11cf4dd4ffe6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:07.728099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84ef3e-4653-4960-b3db-11cf4dd4ffe6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:07.729700 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84ef3e-4653-4960-b3db-11cf4dd4ffe6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:07.739847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-ab84ef3e-4653-4960-b3db-11cf4dd4ffe6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:14:07.739847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:07.739847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:14:07.739847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:14:07.739847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:14:07.739847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:14:07.739847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:14:07.739847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:07.739847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:14:07.739847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:07.739847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:07.739847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:07.739847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:07.739847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:07.739847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:07.739847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:07.739847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:07.740899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:07.740899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:07.740899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:07.740899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:07.740899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:07.740899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:07.740899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:14:07.740899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:14:07.740899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:14:07.740899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:14:07.740899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:07.740899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:07.740899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:07.740899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:07.740899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:07.740899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:07.741705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:07.741705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:07.741705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:07.741705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:07.741705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:07.741705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:07.741705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:07.741705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:07.741705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:07.741705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:07.741705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:14:07.741705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:07.741705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:14:07.741705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:07.748985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 459/912] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:14:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:07.750498 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84ef3e-4653-4960-b3db-11cf4dd4ffe6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:07.780916 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79713c-ce05-48be-8d9c-7bed7598b777 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:07.819298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad89a8-7197-4beb-9329-0073eb7c80e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:07.819928 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad89a8-7197-4beb-9329-0073eb7c80e8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:07.820205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad89a8-7197-4beb-9329-0073eb7c80e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:07.820472 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad89a8-7197-4beb-9329-0073eb7c80e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:07.827858 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad89a8-7197-4beb-9329-0073eb7c80e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:07.829876 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad89a8-7197-4beb-9329-0073eb7c80e8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:07.841795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-baad89a8-7197-4beb-9329-0073eb7c80e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:07.841795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:07.841795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:07.841795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:07.841795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:07.841795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:07.841795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:07.841795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:07.841795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:07.841795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:07.841795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:07.841795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:07.841795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:07.841795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:07.841795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:07.842935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:07.842935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:07.842935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:07.842935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:07.842935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:07.842935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:07.842935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:07.842935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:07.842935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:07.842935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:07.842935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:07.842935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:07.842935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:07.842935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:07.842935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:07.842935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:07.842935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:07.844038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:07.844038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:07.844038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:07.844038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:07.844038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:07.844038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:07.844038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:07.844038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:07.845281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 454/913] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:07.850659 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad89a8-7197-4beb-9329-0073eb7c80e8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:07.864078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:07.869766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84ef3e-4653-4960-b3db-11cf4dd4ffe6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:07.885848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:07.911509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0c305-82e1-478c-867a-d1e84b7589a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:07.912149 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0c305-82e1-478c-867a-d1e84b7589a3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:07.912432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0c305-82e1-478c-867a-d1e84b7589a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:07.912707 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0c305-82e1-478c-867a-d1e84b7589a3 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:07.913448 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0c305-82e1-478c-867a-d1e84b7589a3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:07.913757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0c305-82e1-478c-867a-d1e84b7589a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:07.915199 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0c305-82e1-478c-867a-d1e84b7589a3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:07.922291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 460/914] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:14:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:07.927342 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0c305-82e1-478c-867a-d1e84b7589a3 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:07.971305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad89a8-7197-4beb-9329-0073eb7c80e8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:08.008044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e773d-db14-466f-8fb2-3368a65ae1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:08.008727 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e773d-db14-466f-8fb2-3368a65ae1ae None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.009257 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e773d-db14-466f-8fb2-3368a65ae1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.009368 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e773d-db14-466f-8fb2-3368a65ae1ae None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.010067 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e773d-db14-466f-8fb2-3368a65ae1ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:08.010455 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e773d-db14-466f-8fb2-3368a65ae1ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:08.011971 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e773d-db14-466f-8fb2-3368a65ae1ae None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:08.029760 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 455/915] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:08.030860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e773d-db14-466f-8fb2-3368a65ae1ae None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:08.073751 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0c305-82e1-478c-867a-d1e84b7589a3 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:08.119427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb158333-3fb9-4b0c-a118-40e84a582b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:08.120165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb158333-3fb9-4b0c-a118-40e84a582b75 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.120279 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb158333-3fb9-4b0c-a118-40e84a582b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.120552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb158333-3fb9-4b0c-a118-40e84a582b75 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.121173 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb158333-3fb9-4b0c-a118-40e84a582b75 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:08.121470 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb158333-3fb9-4b0c-a118-40e84a582b75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:08.122855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb158333-3fb9-4b0c-a118-40e84a582b75 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:08.128447 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:08.132804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 461/916] 10.222.0.39 () {56 vars in 1114 bytes} [Mon Aug 8 02:14:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:08.134208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb158333-3fb9-4b0c-a118-40e84a582b75 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:08.143642 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e773d-db14-466f-8fb2-3368a65ae1ae None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:08.162369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:08.192053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f99830-4149-4ef4-bb87-cf23b0223dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:08.192663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f99830-4149-4ef4-bb87-cf23b0223dee None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.192940 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f99830-4149-4ef4-bb87-cf23b0223dee None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.193212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f99830-4149-4ef4-bb87-cf23b0223dee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.193941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f99830-4149-4ef4-bb87-cf23b0223dee None admin] RBAC: Adding query filter params (role.id=ade71d0856a946a0a91f6097a102aa7c, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=b88ccd3e5c4c48d3a27f4eaa7874e28e) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:08.194277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f99830-4149-4ef4-bb87-cf23b0223dee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:08.195713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f99830-4149-4ef4-bb87-cf23b0223dee None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:08.201240 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 456/917] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:08 2022] GET /identity/v3/role_assignments?user.id=b88ccd3e5c4c48d3a27f4eaa7874e28e&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=ade71d0856a946a0a91f6097a102aa7c => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:08.202446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b55dc-ac63-426b-81b4-587765f68d20 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.202707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b55dc-ac63-426b-81b4-587765f68d20 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.202982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b55dc-ac63-426b-81b4-587765f68d20 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.204299 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 457/918] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:08.210129 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f791750-0ba1-420d-8a60-9e947f021b6f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.210480 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f791750-0ba1-420d-8a60-9e947f021b6f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.210715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f791750-0ba1-420d-8a60-9e947f021b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.234571 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4f791750-0ba1-420d-8a60-9e947f021b6f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:08.235826 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4f791750-0ba1-420d-8a60-9e947f021b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:08.276203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb158333-3fb9-4b0c-a118-40e84a582b75 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:08.318449 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 458/919] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:08.329473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045574cf-e6c5-4669-861a-5fdc7224afeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:08.330383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045574cf-e6c5-4669-861a-5fdc7224afeb None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.330805 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045574cf-e6c5-4669-861a-5fdc7224afeb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.331219 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045574cf-e6c5-4669-861a-5fdc7224afeb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.332181 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045574cf-e6c5-4669-861a-5fdc7224afeb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:08.333789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045574cf-e6c5-4669-861a-5fdc7224afeb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:08.338302 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f791750-0ba1-420d-8a60-9e947f021b6f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:08.368839 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 462/920] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:14:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:14:08.439547 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:08.467528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f791750-0ba1-420d-8a60-9e947f021b6f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:08.513921 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ec03b-2d29-42a6-8cac-29124060f39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:08.514976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ec03b-2d29-42a6-8cac-29124060f39b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.515486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ec03b-2d29-42a6-8cac-29124060f39b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.516021 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ec03b-2d29-42a6-8cac-29124060f39b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.516892 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ec03b-2d29-42a6-8cac-29124060f39b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:08.518592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ec03b-2d29-42a6-8cac-29124060f39b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:08.529584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-2d6ec03b-2d29-42a6-8cac-29124060f39b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:14:08.529584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:08.529584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:14:08.529584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:14:08.529584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:14:08.529584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:14:08.529584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:14:08.529584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:08.529584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:14:08.529584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:08.529584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:08.529584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:08.529584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:08.529584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:08.529584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:08.529584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:08.529584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:08.530690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:08.530690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:08.530690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:08.530690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:08.530690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:08.530690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:08.530690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:14:08.530690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:14:08.530690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:14:08.530690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:14:08.530690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:08.530690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:08.530690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:08.530690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:08.530690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:08.530690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:08.531489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:08.531489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:08.531489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:08.531489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:08.531489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:08.531489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:08.531489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:08.531489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:08.531489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:08.531489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:08.531489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:14:08.531489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:08.531489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:14:08.531489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:08.536390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 459/921] 10.222.0.39 () {56 vars in 1126 bytes} [Mon Aug 8 02:14:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:08.537450 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045574cf-e6c5-4669-861a-5fdc7224afeb None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:08.631307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:08.656282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045574cf-e6c5-4669-861a-5fdc7224afeb None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:08.693201 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186208f4-c71a-4f01-bfd9-d738ce616a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:08.693819 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186208f4-c71a-4f01-bfd9-d738ce616a59 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.694030 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186208f4-c71a-4f01-bfd9-d738ce616a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.694281 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186208f4-c71a-4f01-bfd9-d738ce616a59 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.694929 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186208f4-c71a-4f01-bfd9-d738ce616a59 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:08.695236 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186208f4-c71a-4f01-bfd9-d738ce616a59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:08.696606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186208f4-c71a-4f01-bfd9-d738ce616a59 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:08.702030 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 463/922] 10.222.0.39 () {56 vars in 1122 bytes} [Mon Aug 8 02:14:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:08.708076 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ec03b-2d29-42a6-8cac-29124060f39b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:08.816790 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ec03b-2d29-42a6-8cac-29124060f39b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:08.852887 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3512c-66b8-4973-a2fd-691e8e1ab47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:08.853871 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3512c-66b8-4973-a2fd-691e8e1ab47b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.854410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3512c-66b8-4973-a2fd-691e8e1ab47b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.854898 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3512c-66b8-4973-a2fd-691e8e1ab47b None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.855792 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3512c-66b8-4973-a2fd-691e8e1ab47b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:08.856319 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3512c-66b8-4973-a2fd-691e8e1ab47b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:08.857895 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3512c-66b8-4973-a2fd-691e8e1ab47b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:08.863598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 460/923] 10.222.0.39 () {56 vars in 1122 bytes} [Mon Aug 8 02:14:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:08.869632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186208f4-c71a-4f01-bfd9-d738ce616a59 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:08.977463 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186208f4-c71a-4f01-bfd9-d738ce616a59 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:09.012636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e4386-2089-43f0-8c2b-3f4e82dcc29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:09.013605 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e4386-2089-43f0-8c2b-3f4e82dcc29e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:09.014099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e4386-2089-43f0-8c2b-3f4e82dcc29e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:09.014603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e4386-2089-43f0-8c2b-3f4e82dcc29e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:09.015519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e4386-2089-43f0-8c2b-3f4e82dcc29e None admin] RBAC: Adding query filter params (interface=public, service_id=90ae4c99f4c84331b2a5c9f129ea1c57, region_id=RegionOne) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:09.016046 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e4386-2089-43f0-8c2b-3f4e82dcc29e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:09.017613 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e4386-2089-43f0-8c2b-3f4e82dcc29e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:09.023976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 464/924] 10.222.0.39 () {56 vars in 1251 bytes} [Mon Aug 8 02:14:08 2022] GET /identity/v3/endpoints?service_id=90ae4c99f4c84331b2a5c9f129ea1c57&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:09.030559 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3512c-66b8-4973-a2fd-691e8e1ab47b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:09.040778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:09.137919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3512c-66b8-4973-a2fd-691e8e1ab47b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:09.158091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d401d1-1253-442f-8878-65f450de5f89 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:09.158408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d401d1-1253-442f-8878-65f450de5f89 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:09.158829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d401d1-1253-442f-8878-65f450de5f89 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:09.160595 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 465/925] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:09.170709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd9c59-2264-4416-9a62-18de0e6e690d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:09.171444 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd9c59-2264-4416-9a62-18de0e6e690d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:09.171875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd9c59-2264-4416-9a62-18de0e6e690d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:09.180239 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20151300-77a5-40c4-b7a8-e8e1f72309dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:09.181038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20151300-77a5-40c4-b7a8-e8e1f72309dd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:09.181460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20151300-77a5-40c4-b7a8-e8e1f72309dd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:09.181859 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20151300-77a5-40c4-b7a8-e8e1f72309dd None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:09.182652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20151300-77a5-40c4-b7a8-e8e1f72309dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:09.184155 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20151300-77a5-40c4-b7a8-e8e1f72309dd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:09.190600 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:09.191863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 461/926] 10.222.0.39 () {56 vars in 1087 bytes} [Mon Aug 8 02:14:09 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:14:09.200770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a0cd9c59-2264-4416-9a62-18de0e6e690d None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:09.202111 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a0cd9c59-2264-4416-9a62-18de0e6e690d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:09.282408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 466/927] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:09.301629 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20151300-77a5-40c4-b7a8-e8e1f72309dd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:09.385837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:09.424801 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20151300-77a5-40c4-b7a8-e8e1f72309dd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:09.461000 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b41b8c-91fa-4597-9d4c-6e0dbb5c7956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:09.461627 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b41b8c-91fa-4597-9d4c-6e0dbb5c7956 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:09.461906 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b41b8c-91fa-4597-9d4c-6e0dbb5c7956 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:09.462176 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b41b8c-91fa-4597-9d4c-6e0dbb5c7956 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:09.462805 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b41b8c-91fa-4597-9d4c-6e0dbb5c7956 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:09.464182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b41b8c-91fa-4597-9d4c-6e0dbb5c7956 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:09.473072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-80b41b8c-91fa-4597-9d4c-6e0dbb5c7956 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:14:09.473072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:09.473072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:14:09.473072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:14:09.473072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:14:09.473072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:14:09.473072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:14:09.473072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:09.473072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:14:09.473072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:09.473072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:09.473072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:09.473072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:09.473072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:09.473072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:09.473072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:09.473980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:09.473980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:09.473980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:09.473980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:09.473980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:09.473980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:09.473980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:09.473980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:14:09.473980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:14:09.473980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:14:09.473980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:14:09.473980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:09.473980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:09.473980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:09.473980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:09.473980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:09.473980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:09.475382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:09.475382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:09.475382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:09.475382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:09.475382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:09.475382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:09.475382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:09.475382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:09.475382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:09.475382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:09.475382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:14:09.475382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:09.475382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:14:09.475382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:09.475382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 462/928] 10.222.0.39 () {56 vars in 1132 bytes} [Mon Aug 8 02:14:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:09.482422 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd9c59-2264-4416-9a62-18de0e6e690d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:09.588776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd9c59-2264-4416-9a62-18de0e6e690d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:09.631431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436573b-b9bc-4276-b2ca-a411dab6419c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:09.632035 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436573b-b9bc-4276-b2ca-a411dab6419c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:09.632309 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436573b-b9bc-4276-b2ca-a411dab6419c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:09.632574 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436573b-b9bc-4276-b2ca-a411dab6419c None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:09.633203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0436573b-b9bc-4276-b2ca-a411dab6419c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:09.633501 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0436573b-b9bc-4276-b2ca-a411dab6419c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:09.634876 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0436573b-b9bc-4276-b2ca-a411dab6419c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:09.639867 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:09.640119 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 467/929] 10.222.0.39 () {56 vars in 1126 bytes} [Mon Aug 8 02:14:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:09.646037 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b41b8c-91fa-4597-9d4c-6e0dbb5c7956 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:09.754018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b41b8c-91fa-4597-9d4c-6e0dbb5c7956 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:09.789331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f3635-9d47-4aaf-a4be-b3b85931b6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:09.789930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f3635-9d47-4aaf-a4be-b3b85931b6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:09.790206 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f3635-9d47-4aaf-a4be-b3b85931b6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:09.790502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f3635-9d47-4aaf-a4be-b3b85931b6a8 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:09.791126 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f3635-9d47-4aaf-a4be-b3b85931b6a8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:09.791424 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f3635-9d47-4aaf-a4be-b3b85931b6a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:09.792789 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f3635-9d47-4aaf-a4be-b3b85931b6a8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:09.794072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:09.802353 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 463/930] 10.222.0.39 () {56 vars in 1126 bytes} [Mon Aug 8 02:14:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:09.804442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436573b-b9bc-4276-b2ca-a411dab6419c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:09.913066 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436573b-b9bc-4276-b2ca-a411dab6419c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:09.948208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6debcd-cc41-4394-be50-c581138ae995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:09.948818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6debcd-cc41-4394-be50-c581138ae995 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:09.949091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6debcd-cc41-4394-be50-c581138ae995 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:09.949357 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6debcd-cc41-4394-be50-c581138ae995 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:09.950023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6debcd-cc41-4394-be50-c581138ae995 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:09.951402 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6debcd-cc41-4394-be50-c581138ae995 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:09.975084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 468/931] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:14:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:14:10.302424 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095b485-7d2d-4ce5-8cf1-c81367127c7c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:10.302711 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095b485-7d2d-4ce5-8cf1-c81367127c7c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:10.303102 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095b485-7d2d-4ce5-8cf1-c81367127c7c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:10.304702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 464/932] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:10.310047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbba1e-555e-40e1-b8e4-ae61b6296ea9 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:10.310363 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbba1e-555e-40e1-b8e4-ae61b6296ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:10.310633 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbba1e-555e-40e1-b8e4-ae61b6296ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:10.329447 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-dbdbba1e-555e-40e1-b8e4-ae61b6296ea9 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:10.330725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dbdbba1e-555e-40e1-b8e4-ae61b6296ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:10.384193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:10.411089 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 469/933] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:10.433623 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095b485-7d2d-4ce5-8cf1-c81367127c7c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:10.520492 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff3ebd-f38c-460c-8d4b-4aa3a7d7611f None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:10.520868 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff3ebd-f38c-460c-8d4b-4aa3a7d7611f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:10.521040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff3ebd-f38c-460c-8d4b-4aa3a7d7611f None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:10.522665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 470/934] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:14:10.527750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3c9c8-a2bf-4c24-9f95-0a03bc3fe40c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:10.528038 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3c9c8-a2bf-4c24-9f95-0a03bc3fe40c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:10.528313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3c9c8-a2bf-4c24-9f95-0a03bc3fe40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:10.548614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7af3c9c8-a2bf-4c24-9f95-0a03bc3fe40c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:10.549820 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7af3c9c8-a2bf-4c24-9f95-0a03bc3fe40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:10.550981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095b485-7d2d-4ce5-8cf1-c81367127c7c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:10.596720 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36137fb-fe36-4906-9b36-f873a374489d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:10.597340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36137fb-fe36-4906-9b36-f873a374489d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:10.597617 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36137fb-fe36-4906-9b36-f873a374489d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:10.597883 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36137fb-fe36-4906-9b36-f873a374489d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:10.612301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36137fb-fe36-4906-9b36-f873a374489d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:10.617024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e36137fb-fe36-4906-9b36-f873a374489d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:14:10.617024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:10.617024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:10.617024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:10.617024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:10.617024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:10.617024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:10.617024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:10.617024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:10.617024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:10.617024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:10.617024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:10.617024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:10.617024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:10.617024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:10.617024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:10.617968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:10.617968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:10.617968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:10.617968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:10.617968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:10.617968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:10.617968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:10.617968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:10.617968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:10.617968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:10.617968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:10.617968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:10.617968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:10.617968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:10.617968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:10.617968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:10.618840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:10.618840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:10.618840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:10.618840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:10.618840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:10.618840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:10.618840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:10.618840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:10.618840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:10.618840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:10.618840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:14:10.618840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:10.618840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 465/935] 10.222.0.39 () {56 vars in 1120 bytes} [Mon Aug 8 02:14:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:10.623844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36137fb-fe36-4906-9b36-f873a374489d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:10.645994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 471/936] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:10.670423 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3c9c8-a2bf-4c24-9f95-0a03bc3fe40c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:10.697180 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:10.752429 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36137fb-fe36-4906-9b36-f873a374489d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:10.802559 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3c9c8-a2bf-4c24-9f95-0a03bc3fe40c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:10.803700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e7ef8-0d72-43e7-b73c-dbd8bbc93077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:10.804313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e7ef8-0d72-43e7-b73c-dbd8bbc93077 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:10.804592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e7ef8-0d72-43e7-b73c-dbd8bbc93077 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:10.804859 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e7ef8-0d72-43e7-b73c-dbd8bbc93077 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:10.805517 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e7ef8-0d72-43e7-b73c-dbd8bbc93077 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:10.805887 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e7ef8-0d72-43e7-b73c-dbd8bbc93077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:10.807602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e7ef8-0d72-43e7-b73c-dbd8bbc93077 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:10.817135 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 466/937] 10.222.0.39 () {56 vars in 1115 bytes} [Mon Aug 8 02:14:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:10.823791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e7ef8-0d72-43e7-b73c-dbd8bbc93077 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:10.825893 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:10.849366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77fe48-4dc8-4de4-8fd5-943ae8dbcaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:10.850214 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77fe48-4dc8-4de4-8fd5-943ae8dbcaf5 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:10.850654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77fe48-4dc8-4de4-8fd5-943ae8dbcaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:10.851076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77fe48-4dc8-4de4-8fd5-943ae8dbcaf5 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:10.851857 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77fe48-4dc8-4de4-8fd5-943ae8dbcaf5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:10.853475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77fe48-4dc8-4de4-8fd5-943ae8dbcaf5 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:10.856284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: WARNING keystone.server.flask.common [None req-fb77fe48-4dc8-4de4-8fd5-943ae8dbcaf5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 08 02:14:10.878218 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.sql.core [None req-fb77fe48-4dc8-4de4-8fd5-943ae8dbcaf5 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 02:14:10.878218 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 02:14:10.878218 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [parameters: {'id': '4ac6bdcc073a47aa9de14b54e1d4731e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 02:14:10.878218 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80159) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 02:14:10.897691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-fb77fe48-4dc8-4de4-8fd5-943ae8dbcaf5 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4ac6bdcc073a47aa9de14b54e1d4731e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4ac6bdcc073a47aa9de14b54e1d4731e. Aug 08 02:14:10.897691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:10.897691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 02:14:10.897691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 02:14:10.897691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 02:14:10.897691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 02:14:10.897691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 02:14:10.897691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 02:14:10.897691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 02:14:10.897691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application conn.query(q) Aug 08 02:14:10.897691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 02:14:10.897691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 02:14:10.897691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 02:14:10.898780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application result.read() Aug 08 02:14:10.898780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 02:14:10.898780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 02:14:10.898780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 02:14:10.898780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 02:14:10.898780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 02:14:10.898780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 02:14:10.898780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 02:14:10.898780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 02:14:10.898780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 02:14:10.898780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:10.898780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 02:14:10.898780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:10.898780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:10.898780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 02:14:10.898780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 02:14:10.898780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 08 02:14:10.898780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 08 02:14:10.899957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 02:14:10.899957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application next(self.gen) Aug 08 02:14:10.899957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 02:14:10.899957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application yield resource Aug 08 02:14:10.899957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 02:14:10.899957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application next(self.gen) Aug 08 02:14:10.899957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 02:14:10.899957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 02:14:10.899957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 02:14:10.899957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 02:14:10.899957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 02:14:10.899957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise self.value Aug 08 02:14:10.899957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 02:14:10.899957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 02:14:10.899957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 02:14:10.899957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application session.commit() Aug 08 02:14:10.899957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 02:14:10.901067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 02:14:10.901067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 02:14:10.901067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 02:14:10.901067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 02:14:10.901067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self.session.flush() Aug 08 02:14:10.901067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 02:14:10.901067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 02:14:10.901067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 02:14:10.901067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 02:14:10.901067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 02:14:10.901067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application compat.raise_( Aug 08 02:14:10.901067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 02:14:10.901067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception Aug 08 02:14:10.901067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 02:14:10.901067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 02:14:10.901067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 02:14:10.901067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 02:14:10.902162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 02:14:10.902162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 02:14:10.902162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 02:14:10.902162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 02:14:10.902162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 02:14:10.902162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 02:14:10.902162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 02:14:10.902162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 02:14:10.902162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 02:14:10.902162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 02:14:10.902162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 02:14:10.902162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 02:14:10.902162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 02:14:10.902162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 02:14:10.902162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 02:14:10.903180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 02:14:10.903180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 02:14:10.903180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception Aug 08 02:14:10.903180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 02:14:10.903180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 02:14:10.903180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 02:14:10.903180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 02:14:10.903180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 02:14:10.903180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 02:14:10.903180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 02:14:10.903180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application conn.query(q) Aug 08 02:14:10.903180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 02:14:10.903180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 02:14:10.903180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 02:14:10.903180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application result.read() Aug 08 02:14:10.903180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 02:14:10.903180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 02:14:10.904371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 02:14:10.904371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 02:14:10.904371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 02:14:10.904371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 02:14:10.904371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 02:14:10.904371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 02:14:10.904371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 02:14:10.904371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 02:14:10.904371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [parameters: {'id': '4ac6bdcc073a47aa9de14b54e1d4731e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 02:14:10.904371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 02:14:10.904371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:10.904371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:10.904371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:10.904371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:10.904371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 08 02:14:10.906041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 08 02:14:10.906041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 02:14:10.906041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 02:14:10.906041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 08 02:14:10.906041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:10.906041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:10.906041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:10.906041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:10.906041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:10.906041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:10.906041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:10.906041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:10.906041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:10.906041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:10.906041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:10.906041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:10.906041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:10.909442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:10.909442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 08 02:14:10.909442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 08 02:14:10.909442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:10.909442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:10.909442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 08 02:14:10.909442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 08 02:14:10.909442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4ac6bdcc073a47aa9de14b54e1d4731e. Aug 08 02:14:10.909442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:10.909442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 472/938] 10.222.0.39 () {60 vars in 1140 bytes} [Mon Aug 8 02:14:10 2022] POST /identity/v3/projects => generated 262 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:10.915502 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77fe48-4dc8-4de4-8fd5-943ae8dbcaf5 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:10.953171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e7ef8-0d72-43e7-b73c-dbd8bbc93077 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:10.974153 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:10.996481 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbdfc4-4e00-4008-ab56-3e26d5e7b8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:10.997221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbdfc4-4e00-4008-ab56-3e26d5e7b8ba None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:10.997566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbdfc4-4e00-4008-ab56-3e26d5e7b8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:10.997939 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbdfc4-4e00-4008-ab56-3e26d5e7b8ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.009799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbdfc4-4e00-4008-ab56-3e26d5e7b8ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:11.011425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbdfc4-4e00-4008-ab56-3e26d5e7b8ba None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:11.026077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f0bbdfc4-4e00-4008-ab56-3e26d5e7b8ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:11.026077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:11.026077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:11.026077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:11.026077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:11.026077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:11.026077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:11.026077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:11.026077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:11.026077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:11.026077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:11.026077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:11.026077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:11.026077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:11.026077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:11.026077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:11.027071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:11.027071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:11.027071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:11.027071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:11.027071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:11.027071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:11.027071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:11.027071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:11.027071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:11.027071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:11.027071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:11.027071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:11.027071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:11.027071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:11.027071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:11.027071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:11.027071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:11.027950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:11.027950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:11.027950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:11.031892 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 467/939] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:11.032803 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbdfc4-4e00-4008-ab56-3e26d5e7b8ba None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:11.049621 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77fe48-4dc8-4de4-8fd5-943ae8dbcaf5 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:11.092755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bd639-6df2-4232-9bb0-33f1f566b005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:11.093570 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bd639-6df2-4232-9bb0-33f1f566b005 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.093684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bd639-6df2-4232-9bb0-33f1f566b005 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.093915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bd639-6df2-4232-9bb0-33f1f566b005 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.102237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bd639-6df2-4232-9bb0-33f1f566b005 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:11.103905 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bd639-6df2-4232-9bb0-33f1f566b005 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:11.116890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-e65bd639-6df2-4232-9bb0-33f1f566b005 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:11.116890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:11.116890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:11.116890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:11.116890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:11.116890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:11.116890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:11.116890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:11.116890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:11.116890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:11.116890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:11.116890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:11.116890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:11.116890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:11.116890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:11.117789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:11.117789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:11.117789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:11.117789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:11.117789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:11.117789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:11.117789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:11.117789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:11.117789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:11.117789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:11.117789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:11.117789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:11.117789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:11.117789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:11.117789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:11.117789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:11.117789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:11.118711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:11.118711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:11.118711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:11.118711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:11.118711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:11.118711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:11.118711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:11.118711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:11.122493 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 473/940] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:11.123873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bd639-6df2-4232-9bb0-33f1f566b005 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:11.156000 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:11.173705 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbdfc4-4e00-4008-ab56-3e26d5e7b8ba None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:11.217966 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94440330-1a68-40ed-95bb-7123f3001253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:11.219361 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94440330-1a68-40ed-95bb-7123f3001253 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.219849 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94440330-1a68-40ed-95bb-7123f3001253 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.220323 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94440330-1a68-40ed-95bb-7123f3001253 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.221185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94440330-1a68-40ed-95bb-7123f3001253 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:11.222666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94440330-1a68-40ed-95bb-7123f3001253 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:11.224590 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94440330-1a68-40ed-95bb-7123f3001253 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:11.234698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bd639-6df2-4232-9bb0-33f1f566b005 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:11.245326 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 468/941] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:11.251200 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94440330-1a68-40ed-95bb-7123f3001253 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:11.275431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462487b-d67e-43e8-b8c5-01d437ebe7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:11.276036 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462487b-d67e-43e8-b8c5-01d437ebe7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.276314 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462487b-d67e-43e8-b8c5-01d437ebe7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.276585 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462487b-d67e-43e8-b8c5-01d437ebe7c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.277300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462487b-d67e-43e8-b8c5-01d437ebe7c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:11.277603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462487b-d67e-43e8-b8c5-01d437ebe7c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:11.279486 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462487b-d67e-43e8-b8c5-01d437ebe7c8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:11.294683 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:11.306695 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 474/942] 10.222.0.39 () {56 vars in 1112 bytes} [Mon Aug 8 02:14:11 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:11.372340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94440330-1a68-40ed-95bb-7123f3001253 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:11.401089 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801bab6-4610-4e60-866e-33bc2ceb6654 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.401423 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801bab6-4610-4e60-866e-33bc2ceb6654 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.401636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801bab6-4610-4e60-866e-33bc2ceb6654 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.407470 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 475/943] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:14:11.409439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29f01d-4118-4714-8b4a-312d40751bf3 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.410460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29f01d-4118-4714-8b4a-312d40751bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.410800 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29f01d-4118-4714-8b4a-312d40751bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.411339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab7936-abeb-4ea9-9b04-db19b2914afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:11.411985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab7936-abeb-4ea9-9b04-db19b2914afe None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.412266 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab7936-abeb-4ea9-9b04-db19b2914afe None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.412539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab7936-abeb-4ea9-9b04-db19b2914afe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.422116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab7936-abeb-4ea9-9b04-db19b2914afe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:11.423628 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab7936-abeb-4ea9-9b04-db19b2914afe None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:11.431364 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-da29f01d-4118-4714-8b4a-312d40751bf3 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:11.433009 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-da29f01d-4118-4714-8b4a-312d40751bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:11.438850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-29ab7936-abeb-4ea9-9b04-db19b2914afe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:14:11.438850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:11.438850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:11.438850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:11.438850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:11.438850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:11.438850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:11.438850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:11.438850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:11.438850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:11.438850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:11.438850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:11.438850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:11.438850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:11.438850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:11.438850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:11.439918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:11.439918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:11.439918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:11.439918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:11.439918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:11.439918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:11.439918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:11.439918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:11.439918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:11.439918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:11.439918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:11.439918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:11.439918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:11.439918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:11.439918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:11.439918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:11.439918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:11.439918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:11.440842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:11.440842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:11.440842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:11.440842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:11.440842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:11.440842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:11.440842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:11.440842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:11.440842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:14:11.440842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:11.440842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 469/944] 10.222.0.39 () {56 vars in 1117 bytes} [Mon Aug 8 02:14:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:11.449556 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab7936-abeb-4ea9-9b04-db19b2914afe None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:11.513339 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 476/945] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:11.541240 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29f01d-4118-4714-8b4a-312d40751bf3 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:11.565735 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:11.568848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:11.579929 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab7936-abeb-4ea9-9b04-db19b2914afe None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:11.620006 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72068e0c-5226-4550-9cf1-7e17b4b9fd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:11.620861 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72068e0c-5226-4550-9cf1-7e17b4b9fd11 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.621322 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72068e0c-5226-4550-9cf1-7e17b4b9fd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.621836 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72068e0c-5226-4550-9cf1-7e17b4b9fd11 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.622914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72068e0c-5226-4550-9cf1-7e17b4b9fd11 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:11.623371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72068e0c-5226-4550-9cf1-7e17b4b9fd11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:11.625111 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72068e0c-5226-4550-9cf1-7e17b4b9fd11 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:11.646927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 470/946] 10.222.0.39 () {56 vars in 1149 bytes} [Mon Aug 8 02:14:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:11.655330 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72068e0c-5226-4550-9cf1-7e17b4b9fd11 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:11.670067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29f01d-4118-4714-8b4a-312d40751bf3 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:11.709211 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ce183-8980-4873-bafd-3965a073c185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:11.709893 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ce183-8980-4873-bafd-3965a073c185 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.710309 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ce183-8980-4873-bafd-3965a073c185 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.710658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ce183-8980-4873-bafd-3965a073c185 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.711326 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ce183-8980-4873-bafd-3965a073c185 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:11.712900 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ce183-8980-4873-bafd-3965a073c185 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:11.722936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-617ce183-8980-4873-bafd-3965a073c185 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:14:11.722936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:11.722936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:14:11.722936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:14:11.722936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:14:11.722936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:14:11.722936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:14:11.722936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:11.722936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:14:11.722936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:11.722936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:11.722936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:11.722936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:11.722936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:11.722936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:11.722936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:11.722936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:11.724142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:11.724142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:11.724142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:11.724142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:11.724142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:11.724142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:11.724142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:14:11.724142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:14:11.724142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:14:11.724142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:14:11.724142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:11.724142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:11.724142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:11.724142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:11.724142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:11.724142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:11.725188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:11.725188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:11.725188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:11.725188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:11.725188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:11.725188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:11.725188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:11.725188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:11.725188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:11.725188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:11.725188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:14:11.725188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:11.725188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:14:11.725188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:11.726786 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 477/947] 10.222.0.39 () {56 vars in 1126 bytes} [Mon Aug 8 02:14:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:11.734310 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ce183-8980-4873-bafd-3965a073c185 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:11.773396 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72068e0c-5226-4550-9cf1-7e17b4b9fd11 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:11.813043 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284746f-7081-48e1-b075-c5373870f298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:11.813828 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284746f-7081-48e1-b075-c5373870f298 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.814204 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284746f-7081-48e1-b075-c5373870f298 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.814605 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284746f-7081-48e1-b075-c5373870f298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.818612 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:11.826797 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5284746f-7081-48e1-b075-c5373870f298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:11.828398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5284746f-7081-48e1-b075-c5373870f298 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:11.836356 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ce183-8980-4873-bafd-3965a073c185 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:11.844546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5284746f-7081-48e1-b075-c5373870f298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:11.844546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:11.844546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:11.844546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:11.844546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:11.844546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:11.844546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:11.844546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:11.844546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:11.844546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:11.844546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:11.844546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:11.844546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:11.844546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:11.844546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:11.844546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:11.845733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:11.845733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:11.845733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:11.845733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:11.845733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:11.845733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:11.845733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:11.845733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:11.845733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:11.845733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:11.845733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:11.845733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:11.845733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:11.845733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:11.845733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:11.845733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:11.845733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:11.846888 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:11.846888 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:11.846888 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:11.852750 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 471/948] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:11.853832 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284746f-7081-48e1-b075-c5373870f298 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:11.856711 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:11.871642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709b2f3-b422-4fb8-bd87-0a826a8fef23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:11.872247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709b2f3-b422-4fb8-bd87-0a826a8fef23 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.872520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709b2f3-b422-4fb8-bd87-0a826a8fef23 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.872784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709b2f3-b422-4fb8-bd87-0a826a8fef23 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.873463 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709b2f3-b422-4fb8-bd87-0a826a8fef23 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:11.873704 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709b2f3-b422-4fb8-bd87-0a826a8fef23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:11.875079 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709b2f3-b422-4fb8-bd87-0a826a8fef23 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:11.885221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 478/949] 10.222.0.39 () {56 vars in 1122 bytes} [Mon Aug 8 02:14:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:11.886492 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709b2f3-b422-4fb8-bd87-0a826a8fef23 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:11.971360 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284746f-7081-48e1-b075-c5373870f298 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:11.987302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709b2f3-b422-4fb8-bd87-0a826a8fef23 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:12.009976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243729c-4f2c-4e60-9f5a-d8b4be2e3c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:12.011038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243729c-4f2c-4e60-9f5a-d8b4be2e3c0c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:12.011518 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243729c-4f2c-4e60-9f5a-d8b4be2e3c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:12.012063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243729c-4f2c-4e60-9f5a-d8b4be2e3c0c None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:12.012907 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243729c-4f2c-4e60-9f5a-d8b4be2e3c0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:12.013413 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243729c-4f2c-4e60-9f5a-d8b4be2e3c0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:12.015831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243729c-4f2c-4e60-9f5a-d8b4be2e3c0c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:12.023236 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc247d-89b0-431e-8541-06b7e6795cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:12.023912 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc247d-89b0-431e-8541-06b7e6795cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:12.024252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc247d-89b0-431e-8541-06b7e6795cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:12.024591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc247d-89b0-431e-8541-06b7e6795cf9 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:12.025284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc247d-89b0-431e-8541-06b7e6795cf9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:12.025650 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc247d-89b0-431e-8541-06b7e6795cf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:12.027098 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc247d-89b0-431e-8541-06b7e6795cf9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:12.036882 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 472/951] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:12.037078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 479/951] 10.222.0.39 () {56 vars in 1122 bytes} [Mon Aug 8 02:14:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:12.037839 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc247d-89b0-431e-8541-06b7e6795cf9 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:12.039499 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243729c-4f2c-4e60-9f5a-d8b4be2e3c0c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:12.050431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:12.106832 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:12.146602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243729c-4f2c-4e60-9f5a-d8b4be2e3c0c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:12.148864 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc247d-89b0-431e-8541-06b7e6795cf9 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:12.183180 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9a4e1-81ad-47c5-92f8-a33f593e858a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:12.183804 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9a4e1-81ad-47c5-92f8-a33f593e858a None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:12.184080 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9a4e1-81ad-47c5-92f8-a33f593e858a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:12.184347 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9a4e1-81ad-47c5-92f8-a33f593e858a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:12.185264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9a4e1-81ad-47c5-92f8-a33f593e858a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:12.186341 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134769f-bfdd-49ef-95f7-9ee2fa76820c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:12.186658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9a4e1-81ad-47c5-92f8-a33f593e858a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:12.186959 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134769f-bfdd-49ef-95f7-9ee2fa76820c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:12.187234 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134769f-bfdd-49ef-95f7-9ee2fa76820c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:12.187554 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134769f-bfdd-49ef-95f7-9ee2fa76820c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:12.195049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134769f-bfdd-49ef-95f7-9ee2fa76820c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:12.196599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134769f-bfdd-49ef-95f7-9ee2fa76820c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:12.207907 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 473/952] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:14:12 2022] POST /identity/v3/endpoints => generated 349 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:14:12.208745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-2134769f-bfdd-49ef-95f7-9ee2fa76820c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:12.208745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:12.208745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:12.208745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:12.208745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:12.208745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:12.208745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:12.208745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:12.208745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:12.208745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:12.208745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:12.208745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:12.208745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:12.208745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:12.208745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:12.209601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:12.209601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:12.209601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:12.209601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:12.209601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:12.209601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:12.209601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:12.209601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:12.209601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:12.209601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:12.209601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:12.209601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:12.209601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:12.209601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:12.209601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:12.209601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:12.209601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:12.216099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:12.216099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:12.216099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:12.216099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:12.216099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:12.216099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:12.216099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:12.216099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:12.216099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 480/953] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:12.216597 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ab822-d74d-497a-9967-8ab73cd86000 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:12.216597 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ab822-d74d-497a-9967-8ab73cd86000 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:12.216597 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ab822-d74d-497a-9967-8ab73cd86000 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:12.219261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 474/954] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:12.219413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134769f-bfdd-49ef-95f7-9ee2fa76820c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:12.220387 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e53bc-6bcc-4a90-8ef3-91913672e896 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:12.220887 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e53bc-6bcc-4a90-8ef3-91913672e896 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:12.221416 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e53bc-6bcc-4a90-8ef3-91913672e896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:12.244126 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-637e53bc-6bcc-4a90-8ef3-91913672e896 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:12.246640 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-637e53bc-6bcc-4a90-8ef3-91913672e896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:12.300103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:12.335838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134769f-bfdd-49ef-95f7-9ee2fa76820c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:12.343110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 475/955] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:12.370348 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e53bc-6bcc-4a90-8ef3-91913672e896 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:12.396573 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ae74d-42c0-4034-9edf-2755a4ab0eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:12.396573 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ae74d-42c0-4034-9edf-2755a4ab0eaa None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:12.396937 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ae74d-42c0-4034-9edf-2755a4ab0eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:12.397078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ae74d-42c0-4034-9edf-2755a4ab0eaa None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:12.397765 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ae74d-42c0-4034-9edf-2755a4ab0eaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:12.398069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ae74d-42c0-4034-9edf-2755a4ab0eaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:12.399524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ae74d-42c0-4034-9edf-2755a4ab0eaa None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:12.416155 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 481/956] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:12.420790 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ae74d-42c0-4034-9edf-2755a4ab0eaa None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:12.432996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:12.502952 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e53bc-6bcc-4a90-8ef3-91913672e896 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:12.543129 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b199d-597f-4da3-8e19-071690b7e256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:12.544037 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b199d-597f-4da3-8e19-071690b7e256 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:12.544472 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b199d-597f-4da3-8e19-071690b7e256 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:12.544890 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b199d-597f-4da3-8e19-071690b7e256 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:12.545879 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b199d-597f-4da3-8e19-071690b7e256 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:12.547479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b199d-597f-4da3-8e19-071690b7e256 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:12.558460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1d9b199d-597f-4da3-8e19-071690b7e256 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:14:12.558460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:12.558460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:14:12.558460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:14:12.558460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:14:12.558460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:14:12.558460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:14:12.558460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:12.558460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:14:12.558460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:12.558460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:12.558460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:12.558460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:12.558460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:12.558460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:12.558460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:12.558460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:12.559904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:12.559904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:12.559904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:12.559904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:12.559904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:12.559904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:12.559904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:14:12.559904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:14:12.559904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:14:12.559904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:14:12.559904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:12.559904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:12.559904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:12.559904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:12.559904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:12.559904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:12.560815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:12.560815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:12.560815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:12.560815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:12.560815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:12.560815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:12.560815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:12.560815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:12.560815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:12.560815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:12.560815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:14:12.560815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:12.560815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:14:12.560815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:12.561584 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ae74d-42c0-4034-9edf-2755a4ab0eaa None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:12.568342 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 476/957] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:12.570092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b199d-597f-4da3-8e19-071690b7e256 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:12.593532 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:12.601339 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd95555-d256-4e6d-b034-c71bee56cea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:12.602155 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd95555-d256-4e6d-b034-c71bee56cea3 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:12.602556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd95555-d256-4e6d-b034-c71bee56cea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:12.602966 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd95555-d256-4e6d-b034-c71bee56cea3 None admin] PATH_INFO: `/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/b88ccd3e5c4c48d3a27f4eaa7874e28e/roles/ade71d0856a946a0a91f6097a102aa7c` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:12.605525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd95555-d256-4e6d-b034-c71bee56cea3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d474b61f1fe4f17aacb7a3665315c79, user_id=b88ccd3e5c4c48d3a27f4eaa7874e28e, role_id=ade71d0856a946a0a91f6097a102aa7c)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:12.607162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd95555-d256-4e6d-b034-c71bee56cea3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:12.630284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 482/958] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:14:12 2022] PUT /identity/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/b88ccd3e5c4c48d3a27f4eaa7874e28e/roles/ade71d0856a946a0a91f6097a102aa7c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:14:12.695097 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b199d-597f-4da3-8e19-071690b7e256 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:12.705509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:12.734284 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e7336-5370-401a-b00f-8a0acc367f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:12.734904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e7336-5370-401a-b00f-8a0acc367f15 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:12.735246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e7336-5370-401a-b00f-8a0acc367f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:12.735530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e7336-5370-401a-b00f-8a0acc367f15 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:12.736160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e7336-5370-401a-b00f-8a0acc367f15 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:12.736461 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e7336-5370-401a-b00f-8a0acc367f15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:12.738505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e7336-5370-401a-b00f-8a0acc367f15 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:12.744475 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 477/959] 10.222.0.39 () {56 vars in 1112 bytes} [Mon Aug 8 02:14:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:12.749950 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd95555-d256-4e6d-b034-c71bee56cea3 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:12.855763 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd95555-d256-4e6d-b034-c71bee56cea3 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:12.890231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a4058-6b68-42e6-9a3f-0ca25f169903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:12.890868 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a4058-6b68-42e6-9a3f-0ca25f169903 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:12.891146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a4058-6b68-42e6-9a3f-0ca25f169903 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:12.891487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a4058-6b68-42e6-9a3f-0ca25f169903 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:12.892120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a4058-6b68-42e6-9a3f-0ca25f169903 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:12.892420 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a4058-6b68-42e6-9a3f-0ca25f169903 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:12.893773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a4058-6b68-42e6-9a3f-0ca25f169903 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:12.899781 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 483/960] 10.222.0.39 () {56 vars in 1112 bytes} [Mon Aug 8 02:14:12 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:13.506382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535f4e1-c237-457f-bd09-9d5ffcd9449e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:13.506894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535f4e1-c237-457f-bd09-9d5ffcd9449e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:13.506972 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535f4e1-c237-457f-bd09-9d5ffcd9449e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:13.508485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 478/961] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:13.513773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcf580-2654-4392-96b9-6f09e1dbf11e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:13.514065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcf580-2654-4392-96b9-6f09e1dbf11e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:13.514375 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcf580-2654-4392-96b9-6f09e1dbf11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:13.532852 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-9ddcf580-2654-4392-96b9-6f09e1dbf11e None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:13.534034 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9ddcf580-2654-4392-96b9-6f09e1dbf11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:13.583740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:13.606590 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 484/962] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:14:13.626574 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535f4e1-c237-457f-bd09-9d5ffcd9449e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:13.744795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535f4e1-c237-457f-bd09-9d5ffcd9449e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:13.779515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66df6f3-9079-43b2-82ac-2bb8fa99d266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:13.780120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66df6f3-9079-43b2-82ac-2bb8fa99d266 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:13.780394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66df6f3-9079-43b2-82ac-2bb8fa99d266 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:13.780660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66df6f3-9079-43b2-82ac-2bb8fa99d266 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:13.781304 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66df6f3-9079-43b2-82ac-2bb8fa99d266 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:13.782757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66df6f3-9079-43b2-82ac-2bb8fa99d266 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:13.785369 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: WARNING keystone.server.flask.common [None req-e66df6f3-9079-43b2-82ac-2bb8fa99d266 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 08 02:14:13.799977 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.sql.core [None req-e66df6f3-9079-43b2-82ac-2bb8fa99d266 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 02:14:13.799977 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 02:14:13.799977 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [parameters: {'user_id': '997c34ec3d274e1ea30e8d72a40597b5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 02:14:13.799977 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80160) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 02:14:13.808003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e66df6f3-9079-43b2-82ac-2bb8fa99d266 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 02:14:13.808003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:13.808003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 02:14:13.808003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 02:14:13.808003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 02:14:13.808003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 02:14:13.808003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 02:14:13.808003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 02:14:13.808003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 02:14:13.808003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application conn.query(q) Aug 08 02:14:13.808003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 02:14:13.808003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 02:14:13.808003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 02:14:13.808003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result.read() Aug 08 02:14:13.808003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 02:14:13.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 02:14:13.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 02:14:13.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 02:14:13.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 02:14:13.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 02:14:13.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 02:14:13.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 02:14:13.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 02:14:13.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:13.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 02:14:13.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:13.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:13.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 02:14:13.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 02:14:13.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 08 02:14:13.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 08 02:14:13.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 02:14:13.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application next(self.gen) Aug 08 02:14:13.810128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 02:14:13.810128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application yield resource Aug 08 02:14:13.810128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 02:14:13.810128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application next(self.gen) Aug 08 02:14:13.810128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 02:14:13.810128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 02:14:13.810128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 02:14:13.810128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 02:14:13.810128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 02:14:13.810128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise self.value Aug 08 02:14:13.810128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 02:14:13.810128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 02:14:13.810128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 02:14:13.810128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application session.commit() Aug 08 02:14:13.810128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 02:14:13.810128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 02:14:13.810128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 02:14:13.811085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 02:14:13.811085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 02:14:13.811085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.session.flush() Aug 08 02:14:13.811085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 02:14:13.811085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 02:14:13.811085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 02:14:13.811085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 02:14:13.811085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 02:14:13.811085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application compat.raise_( Aug 08 02:14:13.811085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 02:14:13.811085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception Aug 08 02:14:13.811085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 02:14:13.811085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 02:14:13.811085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 02:14:13.811085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 02:14:13.811085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 02:14:13.811085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 02:14:13.811974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 02:14:13.811974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 02:14:13.811974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 08 02:14:13.811974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 08 02:14:13.811974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 02:14:13.811974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 02:14:13.811974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 02:14:13.811974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 02:14:13.811974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 02:14:13.811974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 02:14:13.811974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 02:14:13.811974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 02:14:13.811974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 02:14:13.811974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 02:14:13.811974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 02:14:13.811974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception Aug 08 02:14:13.812833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 02:14:13.812833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 02:14:13.812833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 02:14:13.812833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 02:14:13.812833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 02:14:13.812833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 02:14:13.812833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 02:14:13.812833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application conn.query(q) Aug 08 02:14:13.812833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 02:14:13.812833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 02:14:13.812833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 02:14:13.812833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result.read() Aug 08 02:14:13.812833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 02:14:13.812833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 02:14:13.812833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 02:14:13.812833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 02:14:13.813673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 02:14:13.813673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 02:14:13.813673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 02:14:13.813673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 02:14:13.813673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 02:14:13.813673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 02:14:13.813673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [parameters: {'user_id': '997c34ec3d274e1ea30e8d72a40597b5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 02:14:13.813673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 02:14:13.813673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:13.813673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:13.813673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:13.813673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:13.813673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:13.813673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:13.813673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:13.818715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:13.818715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:13.818715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:13.818715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:13.818715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:13.818715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:13.818715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:13.818715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 08 02:14:13.818715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 08 02:14:13.818715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:13.818715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:13.818715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:13.818715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:13.818715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:13.818715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:13.818715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 08 02:14:13.818715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 08 02:14:13.819846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 08 02:14:13.819846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 08 02:14:13.819846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 02:14:13.819846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 02:14:13.819846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 02:14:13.819846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:13.819846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 479/963] 10.222.0.39 () {60 vars in 1131 bytes} [Mon Aug 8 02:14:13 2022] POST /identity/v3/users => generated 160 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:13.820326 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcf580-2654-4392-96b9-6f09e1dbf11e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:13.923443 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcf580-2654-4392-96b9-6f09e1dbf11e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:13.959987 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813d196-2d23-45fe-9a81-7ff5059bab91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:13.960777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813d196-2d23-45fe-9a81-7ff5059bab91 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:13.961151 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813d196-2d23-45fe-9a81-7ff5059bab91 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:13.961521 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813d196-2d23-45fe-9a81-7ff5059bab91 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:13.969205 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813d196-2d23-45fe-9a81-7ff5059bab91 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:13.970831 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813d196-2d23-45fe-9a81-7ff5059bab91 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:13.982606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-b813d196-2d23-45fe-9a81-7ff5059bab91 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:14:13.982606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:13.982606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:13.982606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:13.982606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:13.982606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:13.982606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:13.982606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:13.982606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:13.982606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:13.982606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:13.982606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:13.982606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:13.982606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:13.982606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:13.982606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:13.983757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:13.983757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:13.983757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:13.983757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:13.983757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:13.983757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:13.983757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:13.983757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:13.983757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:13.983757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:13.983757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:13.983757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:13.983757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:13.983757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:13.983757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:13.983757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:13.983757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:13.983757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:13.984993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:13.984993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:13.984993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:13.984993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:13.984993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:13.984993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:13.984993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:13.984993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:13.984993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:14:13.984993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:13.990628 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 485/964] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:13.993137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66df6f3-9079-43b2-82ac-2bb8fa99d266 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:14.029061 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:14.102551 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66df6f3-9079-43b2-82ac-2bb8fa99d266 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:14.137546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c51cf-2b67-4051-bea9-79e0b4336ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:14.138146 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c51cf-2b67-4051-bea9-79e0b4336ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.138514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c51cf-2b67-4051-bea9-79e0b4336ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.138714 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c51cf-2b67-4051-bea9-79e0b4336ad0 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.139396 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c51cf-2b67-4051-bea9-79e0b4336ad0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:14.139695 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c51cf-2b67-4051-bea9-79e0b4336ad0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:14.141325 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c51cf-2b67-4051-bea9-79e0b4336ad0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:14.180149 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 480/965] 10.222.0.39 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:13 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:14.391264 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a72216-99a6-4f53-b1bb-af11886d9524 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.391556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a72216-99a6-4f53-b1bb-af11886d9524 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.391717 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a72216-99a6-4f53-b1bb-af11886d9524 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.393341 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 486/966] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:14.399312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08608a-667b-4be0-a812-fd263e9b7368 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.399577 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08608a-667b-4be0-a812-fd263e9b7368 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.399855 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08608a-667b-4be0-a812-fd263e9b7368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.419402 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6a08608a-667b-4be0-a812-fd263e9b7368 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:14.420915 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6a08608a-667b-4be0-a812-fd263e9b7368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:14.500844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 481/967] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:14.523633 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a72216-99a6-4f53-b1bb-af11886d9524 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:14.570661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:14.627064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a72216-99a6-4f53-b1bb-af11886d9524 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:14.661233 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729d092-ed71-4933-bfbc-a59720d2ebda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:14.661774 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729d092-ed71-4933-bfbc-a59720d2ebda None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.662046 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729d092-ed71-4933-bfbc-a59720d2ebda None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.662340 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729d092-ed71-4933-bfbc-a59720d2ebda None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.662921 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729d092-ed71-4933-bfbc-a59720d2ebda None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:14.664261 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729d092-ed71-4933-bfbc-a59720d2ebda None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:14.672941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-a729d092-ed71-4933-bfbc-a59720d2ebda None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:14:14.672941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:14.672941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:14:14.672941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:14:14.672941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:14:14.672941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:14:14.672941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:14:14.672941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:14.672941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:14:14.672941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:14.672941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:14.672941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:14.672941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:14.672941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:14.672941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:14.672941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:14.672941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:14.673976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:14.673976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:14.673976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:14.673976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:14.673976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:14.673976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:14.673976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:14:14.673976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:14:14.673976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:14:14.673976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:14:14.673976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:14.673976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:14.673976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:14.673976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:14.673976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:14.673976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:14.674914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:14.674914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:14.674914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:14.674914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:14.674914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:14.674914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:14.674914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:14.674914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:14.674914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:14.674914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:14.674914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:14:14.674914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:14.674914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:14:14.674914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:14.676373 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 487/968] 10.222.0.39 () {56 vars in 1126 bytes} [Mon Aug 8 02:14:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:14.680545 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08608a-667b-4be0-a812-fd263e9b7368 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:14.704518 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:14.749609 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1c4e5-c823-402a-9f12-8b7f31953431 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.750520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1c4e5-c823-402a-9f12-8b7f31953431 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.750805 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1c4e5-c823-402a-9f12-8b7f31953431 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.752263 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 488/969] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:14:14.756885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc547b-4ad9-4c7a-916c-3f875daecc0c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.757172 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc547b-4ad9-4c7a-916c-3f875daecc0c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.757443 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc547b-4ad9-4c7a-916c-3f875daecc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.775558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-bedc547b-4ad9-4c7a-916c-3f875daecc0c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:14.776754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bedc547b-4ad9-4c7a-916c-3f875daecc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:14.785511 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08608a-667b-4be0-a812-fd263e9b7368 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:14.820651 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf41366-6da9-412f-977e-231319607453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:14.821247 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf41366-6da9-412f-977e-231319607453 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.821511 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf41366-6da9-412f-977e-231319607453 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.821780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf41366-6da9-412f-977e-231319607453 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.822427 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf41366-6da9-412f-977e-231319607453 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:14.822726 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf41366-6da9-412f-977e-231319607453 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:14.824085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf41366-6da9-412f-977e-231319607453 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:14.829224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 482/970] 10.222.0.39 () {56 vars in 1122 bytes} [Mon Aug 8 02:14:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:14.835050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf41366-6da9-412f-977e-231319607453 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:14.848758 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 489/971] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:14:14.871093 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc547b-4ad9-4c7a-916c-3f875daecc0c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:14.912762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:14.920373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:14.936503 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf41366-6da9-412f-977e-231319607453 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:14.971696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b69ab9-e4d4-4cec-b380-b342aba12045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:14.972317 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b69ab9-e4d4-4cec-b380-b342aba12045 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.972592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b69ab9-e4d4-4cec-b380-b342aba12045 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.972856 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b69ab9-e4d4-4cec-b380-b342aba12045 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.973473 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b69ab9-e4d4-4cec-b380-b342aba12045 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:14.973775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b69ab9-e4d4-4cec-b380-b342aba12045 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:14.975160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b69ab9-e4d4-4cec-b380-b342aba12045 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:14.977194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc547b-4ad9-4c7a-916c-3f875daecc0c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:14.979880 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 483/972] 10.222.0.39 () {56 vars in 1122 bytes} [Mon Aug 8 02:14:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:14.981036 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d35fee-9e12-4f40-a28b-598e2702151c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.981318 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d35fee-9e12-4f40-a28b-598e2702151c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.981591 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d35fee-9e12-4f40-a28b-598e2702151c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.982812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 484/973] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:14.985445 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d35fee-9e12-4f40-a28b-598e2702151c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:15.016058 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a99746-95ef-4697-bc43-072d0c23fc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:15.016678 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a99746-95ef-4697-bc43-072d0c23fc32 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:15.017003 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a99746-95ef-4697-bc43-072d0c23fc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:15.017223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a99746-95ef-4697-bc43-072d0c23fc32 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:15.027735 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a99746-95ef-4697-bc43-072d0c23fc32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:15.032389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-12a99746-95ef-4697-bc43-072d0c23fc32 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:14:15.032389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:15.032389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:15.032389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:15.032389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:15.032389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:15.032389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:15.032389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:15.032389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:15.032389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:15.032389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:15.032389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:15.032389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:15.032389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:15.032389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:15.032389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:15.033393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:15.033393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:15.033393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:15.033393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:15.033393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:15.033393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:15.033393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:15.033393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:15.033393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:15.033393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:15.033393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:15.033393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:15.033393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:15.033393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:15.033393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:15.033393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:15.039815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:15.039815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:15.039815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:15.039815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:15.039815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:15.039815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:15.039815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:15.039815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:15.039815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:15.039815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:15.039815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:14:15.039815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:15.039815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 490/974] 10.222.0.39 () {56 vars in 1120 bytes} [Mon Aug 8 02:14:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:15.039815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809b7a6-e17f-4640-a9d6-64304d4ee241 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:15.040765 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809b7a6-e17f-4640-a9d6-64304d4ee241 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:15.040765 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809b7a6-e17f-4640-a9d6-64304d4ee241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:15.055058 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7809b7a6-e17f-4640-a9d6-64304d4ee241 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:15.056259 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7809b7a6-e17f-4640-a9d6-64304d4ee241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:15.090604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d35fee-9e12-4f40-a28b-598e2702151c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:15.122949 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 491/975] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:15.123785 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809b7a6-e17f-4640-a9d6-64304d4ee241 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:15.124202 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b5729-f2d6-47f3-b5ad-a19d79a530ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:15.124971 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b5729-f2d6-47f3-b5ad-a19d79a530ce None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:15.125330 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b5729-f2d6-47f3-b5ad-a19d79a530ce None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:15.125703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b5729-f2d6-47f3-b5ad-a19d79a530ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:15.126507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b5729-f2d6-47f3-b5ad-a19d79a530ce None admin] RBAC: Adding query filter params (interface=admin, service_id=90ae4c99f4c84331b2a5c9f129ea1c57, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:15.126902 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b5729-f2d6-47f3-b5ad-a19d79a530ce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:15.129200 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b5729-f2d6-47f3-b5ad-a19d79a530ce None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:15.134431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 485/976] 10.222.0.39 () {56 vars in 1249 bytes} [Mon Aug 8 02:14:14 2022] GET /identity/v3/endpoints?service_id=90ae4c99f4c84331b2a5c9f129ea1c57&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:15.139699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b5729-f2d6-47f3-b5ad-a19d79a530ce None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:15.156874 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:15.161040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:15.232081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809b7a6-e17f-4640-a9d6-64304d4ee241 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:15.242821 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b5729-f2d6-47f3-b5ad-a19d79a530ce None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:15.266106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef367d21-3ba8-41cc-948e-d76f7c4c0502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:15.266795 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef367d21-3ba8-41cc-948e-d76f7c4c0502 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:15.267069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef367d21-3ba8-41cc-948e-d76f7c4c0502 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:15.267339 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef367d21-3ba8-41cc-948e-d76f7c4c0502 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:15.267990 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef367d21-3ba8-41cc-948e-d76f7c4c0502 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:15.268290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef367d21-3ba8-41cc-948e-d76f7c4c0502 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:15.269636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef367d21-3ba8-41cc-948e-d76f7c4c0502 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:15.276607 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c649b-cb1a-444e-97ce-6a13bd8b357f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:15.277202 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c649b-cb1a-444e-97ce-6a13bd8b357f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:15.277479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c649b-cb1a-444e-97ce-6a13bd8b357f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:15.277751 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c649b-cb1a-444e-97ce-6a13bd8b357f None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:15.277873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 492/977] 10.222.0.39 () {56 vars in 1115 bytes} [Mon Aug 8 02:14:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:15.278382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c649b-cb1a-444e-97ce-6a13bd8b357f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:15.278570 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef367d21-3ba8-41cc-948e-d76f7c4c0502 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:15.279737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c649b-cb1a-444e-97ce-6a13bd8b357f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:15.298276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 486/978] 10.222.0.39 () {56 vars in 1087 bytes} [Mon Aug 8 02:14:15 2022] GET /identity/v3/services => generated 1497 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:14:15.298276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c649b-cb1a-444e-97ce-6a13bd8b357f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:15.376393 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:15.394792 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:15.402196 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c649b-cb1a-444e-97ce-6a13bd8b357f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:15.406410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef367d21-3ba8-41cc-948e-d76f7c4c0502 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:15.448370 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253dee0-366a-4ba8-b204-9088395a238b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:15.449437 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253dee0-366a-4ba8-b204-9088395a238b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:15.449437 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253dee0-366a-4ba8-b204-9088395a238b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:15.449701 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253dee0-366a-4ba8-b204-9088395a238b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:15.451739 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba24d2-0c3c-4ea8-83d6-e02e9371af1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:15.452332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba24d2-0c3c-4ea8-83d6-e02e9371af1b None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:15.452609 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba24d2-0c3c-4ea8-83d6-e02e9371af1b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:15.452878 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba24d2-0c3c-4ea8-83d6-e02e9371af1b None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:15.453524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba24d2-0c3c-4ea8-83d6-e02e9371af1b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:15.455201 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba24d2-0c3c-4ea8-83d6-e02e9371af1b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:15.461486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253dee0-366a-4ba8-b204-9088395a238b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:15.463326 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253dee0-366a-4ba8-b204-9088395a238b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:15.471481 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 493/979] 10.222.0.39 () {60 vars in 1141 bytes} [Mon Aug 8 02:14:15 2022] POST /identity/v3/services => generated 242 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:14:15.485477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6253dee0-366a-4ba8-b204-9088395a238b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:15.485477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:15.485477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:15.485477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:15.485477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:15.485477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:15.485477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:15.485477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:15.485477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:15.485477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:15.485477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:15.485477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:15.485477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:15.485477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:15.485477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:15.485477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:15.486689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:15.486689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:15.486689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:15.486689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:15.486689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:15.486689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:15.486689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:15.486689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:15.486689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:15.486689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:15.486689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:15.486689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:15.486689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:15.486689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:15.486689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:15.486689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:15.486689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:15.488426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:15.488426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:15.488426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:15.488426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 487/980] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:15.493052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba24d2-0c3c-4ea8-83d6-e02e9371af1b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:15.613474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba24d2-0c3c-4ea8-83d6-e02e9371af1b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:15.648719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da9e08-5623-4b54-88d2-b8f2a10ca732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:15.649327 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da9e08-5623-4b54-88d2-b8f2a10ca732 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:15.649602 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da9e08-5623-4b54-88d2-b8f2a10ca732 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:15.649872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da9e08-5623-4b54-88d2-b8f2a10ca732 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:15.650583 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da9e08-5623-4b54-88d2-b8f2a10ca732 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:15.650885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da9e08-5623-4b54-88d2-b8f2a10ca732 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:15.652236 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da9e08-5623-4b54-88d2-b8f2a10ca732 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:15.667362 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 494/981] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:15.672913 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253dee0-366a-4ba8-b204-9088395a238b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:15.776457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253dee0-366a-4ba8-b204-9088395a238b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:15.805982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:15.813098 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c924c7d-3612-4c48-90d0-d85f4d1de823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:15.813728 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c924c7d-3612-4c48-90d0-d85f4d1de823 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:15.814006 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c924c7d-3612-4c48-90d0-d85f4d1de823 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:15.814312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c924c7d-3612-4c48-90d0-d85f4d1de823 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:15.822153 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c924c7d-3612-4c48-90d0-d85f4d1de823 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:15.823629 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c924c7d-3612-4c48-90d0-d85f4d1de823 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:15.835515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-2c924c7d-3612-4c48-90d0-d85f4d1de823 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:14:15.835515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:15.835515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:15.835515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:15.835515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:15.835515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:15.835515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:15.835515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:15.835515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:15.835515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:15.835515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:15.835515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:15.835515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:15.835515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:15.835515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:15.835515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:15.836724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:15.836724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:15.836724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:15.836724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:15.836724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:15.836724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:15.836724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:15.836724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:15.836724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:15.836724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:15.836724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:15.836724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:15.836724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:15.836724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:15.836724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:15.836724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:15.836724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:15.836724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:15.837700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:15.837700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:15.837700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:15.837700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:15.837700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:15.837700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:15.837700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:15.837700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:15.837700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:14:15.837700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:15.838705 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 488/982] 10.222.0.39 () {56 vars in 1117 bytes} [Mon Aug 8 02:14:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:15.841868 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da9e08-5623-4b54-88d2-b8f2a10ca732 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:15.851547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:15.945576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da9e08-5623-4b54-88d2-b8f2a10ca732 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:15.980200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe4ea3-1e2f-4e2c-bd4f-0960130319d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:15.980829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe4ea3-1e2f-4e2c-bd4f-0960130319d3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:15.981083 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe4ea3-1e2f-4e2c-bd4f-0960130319d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:15.981343 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe4ea3-1e2f-4e2c-bd4f-0960130319d3 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:15.982050 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe4ea3-1e2f-4e2c-bd4f-0960130319d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:15.982379 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe4ea3-1e2f-4e2c-bd4f-0960130319d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:15.983823 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe4ea3-1e2f-4e2c-bd4f-0960130319d3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:16.005664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 495/983] 10.222.0.39 () {56 vars in 1149 bytes} [Mon Aug 8 02:14:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:16.007155 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c924c7d-3612-4c48-90d0-d85f4d1de823 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:16.108661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c924c7d-3612-4c48-90d0-d85f4d1de823 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:16.141166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff42216-fcd3-4a49-8da1-0170aa4e7be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:16.141763 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff42216-fcd3-4a49-8da1-0170aa4e7be8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:16.142038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff42216-fcd3-4a49-8da1-0170aa4e7be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:16.142332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff42216-fcd3-4a49-8da1-0170aa4e7be8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:16.199656 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:16.211606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff42216-fcd3-4a49-8da1-0170aa4e7be8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:16.213057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff42216-fcd3-4a49-8da1-0170aa4e7be8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:16.313949 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 489/984] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:16 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:14:16.315443 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe4ea3-1e2f-4e2c-bd4f-0960130319d3 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:16.377556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:16.423136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe4ea3-1e2f-4e2c-bd4f-0960130319d3 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:16.463159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe527f1-be44-4564-8075-7055e89e14a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:16.463915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe527f1-be44-4564-8075-7055e89e14a1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:16.464287 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe527f1-be44-4564-8075-7055e89e14a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:16.464656 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe527f1-be44-4564-8075-7055e89e14a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:16.475009 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe527f1-be44-4564-8075-7055e89e14a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:16.476561 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe527f1-be44-4564-8075-7055e89e14a1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:16.490753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-8fe527f1-be44-4564-8075-7055e89e14a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:16.490753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:16.490753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:16.490753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:16.490753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:16.490753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:16.490753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:16.490753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:16.490753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:16.490753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:16.490753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:16.490753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:16.490753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:16.490753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:16.490753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:16.490753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:16.491877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:16.491877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:16.491877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:16.491877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:16.491877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:16.491877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:16.491877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:16.491877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:16.491877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:16.491877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:16.491877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:16.491877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:16.491877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:16.491877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:16.491877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:16.491877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:16.491877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:16.492985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:16.492985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:16.492985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:16.498585 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 496/985] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:16.499807 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff42216-fcd3-4a49-8da1-0170aa4e7be8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:16.592163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:16.603780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff42216-fcd3-4a49-8da1-0170aa4e7be8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:16.637734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12749397-156d-40f8-95a6-c102b0defc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:16.638301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12749397-156d-40f8-95a6-c102b0defc37 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:16.638562 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12749397-156d-40f8-95a6-c102b0defc37 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:16.638833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12749397-156d-40f8-95a6-c102b0defc37 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:16.639468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12749397-156d-40f8-95a6-c102b0defc37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:16.639771 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12749397-156d-40f8-95a6-c102b0defc37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:16.641133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12749397-156d-40f8-95a6-c102b0defc37 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:16.657432 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 490/986] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:16.658731 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe527f1-be44-4564-8075-7055e89e14a1 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:16.759963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe527f1-be44-4564-8075-7055e89e14a1 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:16.775582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:16.794402 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df2fda-e3bb-4663-a96a-2a20a8dcd5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:16.795026 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df2fda-e3bb-4663-a96a-2a20a8dcd5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:16.795304 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df2fda-e3bb-4663-a96a-2a20a8dcd5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:16.795575 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df2fda-e3bb-4663-a96a-2a20a8dcd5f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:16.802659 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df2fda-e3bb-4663-a96a-2a20a8dcd5f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:16.804080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df2fda-e3bb-4663-a96a-2a20a8dcd5f3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:16.815163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-00df2fda-e3bb-4663-a96a-2a20a8dcd5f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:16.815163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:16.815163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:16.815163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:16.815163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:16.815163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:16.815163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:16.815163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:16.815163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:16.815163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:16.815163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:16.815163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:16.815163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:16.815163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:16.815163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:16.816658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:16.816658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:16.816658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:16.816658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:16.816658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:16.816658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:16.816658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:16.816658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:16.816658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:16.816658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:16.816658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:16.816658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:16.816658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:16.816658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:16.816658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:16.816658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:16.816658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:16.817538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:16.817538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:16.817538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:16.817538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:16.817538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:16.817538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:16.817538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:16.817538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:16.821663 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 497/987] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:16.822919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12749397-156d-40f8-95a6-c102b0defc37 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:16.924400 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12749397-156d-40f8-95a6-c102b0defc37 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:16.959105 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010b363-8cb7-4bb9-bed2-4a8819937028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:16.959709 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010b363-8cb7-4bb9-bed2-4a8819937028 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:16.959987 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010b363-8cb7-4bb9-bed2-4a8819937028 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:16.960260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010b363-8cb7-4bb9-bed2-4a8819937028 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:16.960947 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010b363-8cb7-4bb9-bed2-4a8819937028 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:16.961253 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010b363-8cb7-4bb9-bed2-4a8819937028 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:16.962717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010b363-8cb7-4bb9-bed2-4a8819937028 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:16.980252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 491/988] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:16.981688 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df2fda-e3bb-4663-a96a-2a20a8dcd5f3 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:17.086936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df2fda-e3bb-4663-a96a-2a20a8dcd5f3 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:17.121119 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de07bed-4ec8-4fc4-b044-ab0c34a2f87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:17.121718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de07bed-4ec8-4fc4-b044-ab0c34a2f87e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:17.121995 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de07bed-4ec8-4fc4-b044-ab0c34a2f87e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:17.122276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de07bed-4ec8-4fc4-b044-ab0c34a2f87e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:17.122996 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de07bed-4ec8-4fc4-b044-ab0c34a2f87e None admin] RBAC: Adding query filter params (role.id=ade71d0856a946a0a91f6097a102aa7c, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=b88ccd3e5c4c48d3a27f4eaa7874e28e) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:17.123304 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de07bed-4ec8-4fc4-b044-ab0c34a2f87e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:17.124743 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de07bed-4ec8-4fc4-b044-ab0c34a2f87e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:17.130654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 498/989] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:16 2022] GET /identity/v3/role_assignments?user.id=b88ccd3e5c4c48d3a27f4eaa7874e28e&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=ade71d0856a946a0a91f6097a102aa7c => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:17.291982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36484ff-9fc4-4dd6-8c11-994f07b28f42 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:17.292263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36484ff-9fc4-4dd6-8c11-994f07b28f42 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:17.292533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36484ff-9fc4-4dd6-8c11-994f07b28f42 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:17.293932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 492/990] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:17.299156 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0e09c-179a-4c7a-bcee-68cf844e012e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:17.299442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0e09c-179a-4c7a-bcee-68cf844e012e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:17.299716 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0e09c-179a-4c7a-bcee-68cf844e012e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:17.321159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-fde0e09c-179a-4c7a-bcee-68cf844e012e None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:17.322408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-fde0e09c-179a-4c7a-bcee-68cf844e012e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:17.357797 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:17.394401 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 499/991] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:17.418098 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36484ff-9fc4-4dd6-8c11-994f07b28f42 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:17.430995 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:17.501417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3693e-5ca0-4aac-9086-7461854712ab None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:17.501744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3693e-5ca0-4aac-9086-7461854712ab None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:17.501964 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3693e-5ca0-4aac-9086-7461854712ab None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:17.503507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 500/992] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:14:17.508262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ef63b-5df0-47f8-b47b-9b534037fc18 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:17.508544 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ef63b-5df0-47f8-b47b-9b534037fc18 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:17.508818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ef63b-5df0-47f8-b47b-9b534037fc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:17.527350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36484ff-9fc4-4dd6-8c11-994f07b28f42 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:17.527538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-002ef63b-5df0-47f8-b47b-9b534037fc18 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:17.528625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-002ef63b-5df0-47f8-b47b-9b534037fc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:17.563692 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51f872-f8b9-49e7-b576-187c1574d715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:17.564242 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51f872-f8b9-49e7-b576-187c1574d715 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:17.564520 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51f872-f8b9-49e7-b576-187c1574d715 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:17.564791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51f872-f8b9-49e7-b576-187c1574d715 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:17.565373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51f872-f8b9-49e7-b576-187c1574d715 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:17.566822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51f872-f8b9-49e7-b576-187c1574d715 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:17.580208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3b51f872-f8b9-49e7-b576-187c1574d715 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:14:17.580208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:17.580208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:14:17.580208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:14:17.580208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:14:17.580208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:14:17.580208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:14:17.580208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:17.580208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:14:17.580208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:17.580208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:17.580208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:17.580208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:17.580208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:17.580208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:17.580208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:17.580208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:17.581261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:17.581261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:17.581261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:17.581261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:17.581261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:17.581261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:17.581261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:14:17.581261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:14:17.581261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:14:17.581261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:14:17.581261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:17.581261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:17.581261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:17.581261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:17.581261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:17.581261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:17.582661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:17.582661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:17.582661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:17.582661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:17.582661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:17.582661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:17.582661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:17.582661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:17.582661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:17.582661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:17.582661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:14:17.582661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:17.582661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:14:17.582661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:17.582661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 493/993] 10.222.0.39 () {56 vars in 1126 bytes} [Mon Aug 8 02:14:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:17.588026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51f872-f8b9-49e7-b576-187c1574d715 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:17.620920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 501/994] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:14:17.657294 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ef63b-5df0-47f8-b47b-9b534037fc18 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:17.691172 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:17.728312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51f872-f8b9-49e7-b576-187c1574d715 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:17.768147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:17.771661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ef63b-5df0-47f8-b47b-9b534037fc18 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:17.772619 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1e4ab-8167-47c9-9c31-e018516a2bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:17.773234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1e4ab-8167-47c9-9c31-e018516a2bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:17.773616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1e4ab-8167-47c9-9c31-e018516a2bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:17.773786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1e4ab-8167-47c9-9c31-e018516a2bb2 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:17.774929 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1e4ab-8167-47c9-9c31-e018516a2bb2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:17.774929 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1e4ab-8167-47c9-9c31-e018516a2bb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:17.776127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1e4ab-8167-47c9-9c31-e018516a2bb2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:17.786867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 494/995] 10.222.0.39 () {56 vars in 1122 bytes} [Mon Aug 8 02:14:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:17.787758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1e4ab-8167-47c9-9c31-e018516a2bb2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:17.812374 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944f474-c8b1-463c-899a-ec918c621e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:17.813546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944f474-c8b1-463c-899a-ec918c621e75 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:17.814124 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944f474-c8b1-463c-899a-ec918c621e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:17.814658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944f474-c8b1-463c-899a-ec918c621e75 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:17.815454 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944f474-c8b1-463c-899a-ec918c621e75 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:17.817723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944f474-c8b1-463c-899a-ec918c621e75 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:17.827081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-6944f474-c8b1-463c-899a-ec918c621e75 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:14:17.827081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:17.827081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:14:17.827081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:14:17.827081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:14:17.827081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:14:17.827081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:14:17.827081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:17.827081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:14:17.827081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:17.827081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:17.827081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:17.827081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:17.827081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:17.827081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:17.827081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:17.827081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:17.828108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:17.828108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:17.828108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:17.828108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:17.828108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:17.828108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:17.828108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:14:17.828108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:14:17.828108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:14:17.828108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:14:17.828108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:17.828108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:17.828108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:17.828108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:17.828108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:17.828108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:17.828941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:17.828941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:17.828941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:17.828941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:17.828941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:17.828941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:17.828941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:17.828941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:17.828941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:17.828941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:17.828941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:14:17.828941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:17.828941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:14:17.828941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:17.832023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 502/996] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:17.838126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944f474-c8b1-463c-899a-ec918c621e75 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:17.898692 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1e4ab-8167-47c9-9c31-e018516a2bb2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:17.933182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822da3c-4ec0-4a28-ac28-b13170b6603a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:17.934081 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822da3c-4ec0-4a28-ac28-b13170b6603a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:17.934387 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822da3c-4ec0-4a28-ac28-b13170b6603a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:17.934896 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822da3c-4ec0-4a28-ac28-b13170b6603a None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:17.935946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822da3c-4ec0-4a28-ac28-b13170b6603a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:17.936251 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822da3c-4ec0-4a28-ac28-b13170b6603a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:17.937867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822da3c-4ec0-4a28-ac28-b13170b6603a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:17.944756 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 495/997] 10.222.0.39 () {56 vars in 1122 bytes} [Mon Aug 8 02:14:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:17.949942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822da3c-4ec0-4a28-ac28-b13170b6603a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:17.953428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:18.001864 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944f474-c8b1-463c-899a-ec918c621e75 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:18.037791 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d79e32-f683-4c62-aaaf-b8cd3786a5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:18.038741 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d79e32-f683-4c62-aaaf-b8cd3786a5eb None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:18.039263 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d79e32-f683-4c62-aaaf-b8cd3786a5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:18.039742 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d79e32-f683-4c62-aaaf-b8cd3786a5eb None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:18.040578 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d79e32-f683-4c62-aaaf-b8cd3786a5eb None admin] RBAC: Adding query filter params (name=compute) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:18.040952 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d79e32-f683-4c62-aaaf-b8cd3786a5eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:18.042415 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d79e32-f683-4c62-aaaf-b8cd3786a5eb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:18.047779 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 503/998] 10.222.0.39 () {56 vars in 1112 bytes} [Mon Aug 8 02:14:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:18.053044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d79e32-f683-4c62-aaaf-b8cd3786a5eb None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:18.076483 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822da3c-4ec0-4a28-ac28-b13170b6603a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:18.083671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:18.113903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d650bc-f81c-424d-aab2-00a3b07648f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:18.114533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d650bc-f81c-424d-aab2-00a3b07648f1 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:18.114817 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d650bc-f81c-424d-aab2-00a3b07648f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:18.115087 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d650bc-f81c-424d-aab2-00a3b07648f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:18.115744 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d650bc-f81c-424d-aab2-00a3b07648f1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:18.117107 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d650bc-f81c-424d-aab2-00a3b07648f1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:18.138424 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 496/999] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:14:17 2022] POST /identity/v3/endpoints => generated 325 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:14:18.175718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d79e32-f683-4c62-aaaf-b8cd3786a5eb None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:18.222013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed9484-146f-44a2-98c4-2a7e058dd9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:18.222947 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed9484-146f-44a2-98c4-2a7e058dd9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:18.223378 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed9484-146f-44a2-98c4-2a7e058dd9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:18.223798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed9484-146f-44a2-98c4-2a7e058dd9d2 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:18.224674 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed9484-146f-44a2-98c4-2a7e058dd9d2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:18.225127 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed9484-146f-44a2-98c4-2a7e058dd9d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:18.226770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed9484-146f-44a2-98c4-2a7e058dd9d2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:18.234318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 504/1000] 10.222.0.39 () {56 vars in 1112 bytes} [Mon Aug 8 02:14:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:18.239958 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d650bc-f81c-424d-aab2-00a3b07648f1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:18.309532 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:18.352704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d650bc-f81c-424d-aab2-00a3b07648f1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:18.387371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58828b49-98ec-4f6b-94a6-552e210021e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:18.387976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58828b49-98ec-4f6b-94a6-552e210021e6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:18.388255 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58828b49-98ec-4f6b-94a6-552e210021e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:18.388522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58828b49-98ec-4f6b-94a6-552e210021e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:18.389212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58828b49-98ec-4f6b-94a6-552e210021e6 None admin] RBAC: Adding query filter params (interface=public, service_id=b4a41d14d46a4b67ad709427e0a2b9f3, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:18.389521 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58828b49-98ec-4f6b-94a6-552e210021e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:18.390909 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58828b49-98ec-4f6b-94a6-552e210021e6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:18.396136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 497/1001] 10.222.0.39 () {56 vars in 1251 bytes} [Mon Aug 8 02:14:18 2022] GET /identity/v3/endpoints?service_id=b4a41d14d46a4b67ad709427e0a2b9f3&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:18.401789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed9484-146f-44a2-98c4-2a7e058dd9d2 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:18.504041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:18.517478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed9484-146f-44a2-98c4-2a7e058dd9d2 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:18.551660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e4a43-d987-496b-b345-c0ae9caaab4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:18.552259 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e4a43-d987-496b-b345-c0ae9caaab4d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:18.552534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e4a43-d987-496b-b345-c0ae9caaab4d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:18.552801 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e4a43-d987-496b-b345-c0ae9caaab4d None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:18.553392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e4a43-d987-496b-b345-c0ae9caaab4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:18.554843 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e4a43-d987-496b-b345-c0ae9caaab4d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:18.566337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 505/1002] 10.222.0.39 () {56 vars in 1087 bytes} [Mon Aug 8 02:14:18 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:14:19.185072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110edb67-77d9-4cd7-88bc-9cd6da4a6c72 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:19.185606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110edb67-77d9-4cd7-88bc-9cd6da4a6c72 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:19.185606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110edb67-77d9-4cd7-88bc-9cd6da4a6c72 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:19.187146 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 498/1003] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:19.192183 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fab9c-1640-47bf-a4e0-2abe678635de None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:19.192563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fab9c-1640-47bf-a4e0-2abe678635de None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:19.192740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fab9c-1640-47bf-a4e0-2abe678635de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:19.211814 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-363fab9c-1640-47bf-a4e0-2abe678635de None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:19.213002 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-363fab9c-1640-47bf-a4e0-2abe678635de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:19.282130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 506/1004] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:19 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:19.304526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110edb67-77d9-4cd7-88bc-9cd6da4a6c72 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:19.408188 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110edb67-77d9-4cd7-88bc-9cd6da4a6c72 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:19.439027 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:19.455752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c12a33-004f-4b76-8a44-b084e9ad223c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:19.457223 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c12a33-004f-4b76-8a44-b084e9ad223c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:19.458035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c12a33-004f-4b76-8a44-b084e9ad223c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:19.458035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c12a33-004f-4b76-8a44-b084e9ad223c None admin] PATH_INFO: `/v3/services/image` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:19.458634 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c12a33-004f-4b76-8a44-b084e9ad223c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:19.460618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c12a33-004f-4b76-8a44-b084e9ad223c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:19.469884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-97c12a33-004f-4b76-8a44-b084e9ad223c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:14:19.469884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:19.469884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:14:19.469884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:14:19.469884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:14:19.469884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:14:19.469884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:14:19.469884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:19.469884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:14:19.469884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:19.469884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:19.469884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:19.469884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:19.469884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:19.469884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:19.469884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:19.469884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:19.471050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:19.471050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:19.471050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:19.471050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:19.471050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:19.471050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:19.471050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:14:19.471050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:14:19.471050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:14:19.471050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:14:19.471050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:19.471050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:19.471050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:19.471050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:19.471050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:19.471050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:19.471962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:19.471962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:19.471962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:19.471962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:19.471962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:19.471962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:19.471962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:19.471962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:19.471962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:19.471962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:19.471962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:14:19.471962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:19.471962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:14:19.471962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:19.471962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 499/1005] 10.222.0.39 () {56 vars in 1105 bytes} [Mon Aug 8 02:14:19 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:19.476924 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fab9c-1640-47bf-a4e0-2abe678635de None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:19.558336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:19.578662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fab9c-1640-47bf-a4e0-2abe678635de None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:19.612981 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7e730-c9a8-4c28-84ba-fe1244494d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:19.613639 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7e730-c9a8-4c28-84ba-fe1244494d99 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:19.613921 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7e730-c9a8-4c28-84ba-fe1244494d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:19.614189 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7e730-c9a8-4c28-84ba-fe1244494d99 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:19.614844 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b7e730-c9a8-4c28-84ba-fe1244494d99 None admin] RBAC: Adding query filter params (name=image) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:19.615145 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b7e730-c9a8-4c28-84ba-fe1244494d99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:19.616499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b7e730-c9a8-4c28-84ba-fe1244494d99 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:19.625942 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 507/1006] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:19.627155 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c12a33-004f-4b76-8a44-b084e9ad223c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:19.736770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c12a33-004f-4b76-8a44-b084e9ad223c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:19.771495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0c65a-6ded-4a9c-9f41-935bfb54bf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:19.772092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0c65a-6ded-4a9c-9f41-935bfb54bf2a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:19.772432 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0c65a-6ded-4a9c-9f41-935bfb54bf2a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:19.772707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0c65a-6ded-4a9c-9f41-935bfb54bf2a None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:19.773330 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0c65a-6ded-4a9c-9f41-935bfb54bf2a None admin] RBAC: Adding query filter params (type=image) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:19.773632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0c65a-6ded-4a9c-9f41-935bfb54bf2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:19.775022 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0c65a-6ded-4a9c-9f41-935bfb54bf2a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:19.786336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 500/1007] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:19 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:20.168340 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12370c08-42bd-45d0-a1e3-18d44adddb9a None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:20.168724 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12370c08-42bd-45d0-a1e3-18d44adddb9a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:20.168930 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12370c08-42bd-45d0-a1e3-18d44adddb9a None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:20.170287 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 508/1008] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:20.175659 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c87c4b-dcfa-4d26-beb2-77d0d7e3211d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:20.176045 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c87c4b-dcfa-4d26-beb2-77d0d7e3211d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:20.176416 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c87c4b-dcfa-4d26-beb2-77d0d7e3211d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:20.194904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-28c87c4b-dcfa-4d26-beb2-77d0d7e3211d None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:20.196081 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-28c87c4b-dcfa-4d26-beb2-77d0d7e3211d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:20.246845 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:20.266393 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 501/1009] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:20 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:20.287925 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12370c08-42bd-45d0-a1e3-18d44adddb9a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:20.388543 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12370c08-42bd-45d0-a1e3-18d44adddb9a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:20.428291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157f8e9-77df-47f3-bbe8-6be907b4c915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:20.428843 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157f8e9-77df-47f3-bbe8-6be907b4c915 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:20.429117 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157f8e9-77df-47f3-bbe8-6be907b4c915 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:20.429443 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157f8e9-77df-47f3-bbe8-6be907b4c915 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:20.430573 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157f8e9-77df-47f3-bbe8-6be907b4c915 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:20.431991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157f8e9-77df-47f3-bbe8-6be907b4c915 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:20.434090 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 509/1010] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:20.439976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c87c4b-dcfa-4d26-beb2-77d0d7e3211d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:20.543536 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c87c4b-dcfa-4d26-beb2-77d0d7e3211d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:20.576779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ff22e-ee39-4f11-b4da-ae169567cac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:20.577375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ff22e-ee39-4f11-b4da-ae169567cac4 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:20.577648 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ff22e-ee39-4f11-b4da-ae169567cac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:20.577910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ff22e-ee39-4f11-b4da-ae169567cac4 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:20.578568 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ff22e-ee39-4f11-b4da-ae169567cac4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:20.580049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ff22e-ee39-4f11-b4da-ae169567cac4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:20.611830 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 502/1011] 10.222.0.39 () {60 vars in 1141 bytes} [Mon Aug 8 02:14:20 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:14:20.613823 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c2b4c-e2df-4474-9d92-768b5522d397 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:20.614117 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c2b4c-e2df-4474-9d92-768b5522d397 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:20.614425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c2b4c-e2df-4474-9d92-768b5522d397 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:20.615961 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 510/1012] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:20.621507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6130afc-550d-488f-b05b-62231d18f0e3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:20.621814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6130afc-550d-488f-b05b-62231d18f0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:20.622070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6130afc-550d-488f-b05b-62231d18f0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:20.641336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a6130afc-550d-488f-b05b-62231d18f0e3 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:20.642616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a6130afc-550d-488f-b05b-62231d18f0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:20.674710 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:20.718122 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 503/1013] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:20 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:20.741457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c2b4c-e2df-4474-9d92-768b5522d397 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:20.758436 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:20.852502 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c2b4c-e2df-4474-9d92-768b5522d397 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:20.886939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b911f08-43ef-44ea-a1b4-63db2ff71018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:20.887585 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b911f08-43ef-44ea-a1b4-63db2ff71018 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:20.887859 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b911f08-43ef-44ea-a1b4-63db2ff71018 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:20.888128 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b911f08-43ef-44ea-a1b4-63db2ff71018 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:20.888720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b911f08-43ef-44ea-a1b4-63db2ff71018 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:20.890086 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b911f08-43ef-44ea-a1b4-63db2ff71018 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:20.899833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-2b911f08-43ef-44ea-a1b4-63db2ff71018 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:14:20.899833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:20.899833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:14:20.899833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:14:20.899833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:14:20.899833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:14:20.899833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:14:20.899833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:20.899833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:14:20.899833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:20.899833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:20.899833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:20.899833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:20.899833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:20.899833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:20.899833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:20.899833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:20.900833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:20.900833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:20.900833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:20.900833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:20.900833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:20.900833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:20.900833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:14:20.900833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:14:20.900833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:14:20.900833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:14:20.900833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:20.900833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:20.900833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:20.900833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:20.900833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:20.900833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:20.901671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:20.901671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:20.901671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:20.901671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:20.901671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:20.901671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:20.901671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:20.901671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:20.901671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:20.901671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:20.901671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:14:20.901671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:20.901671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:14:20.901671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:20.901671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 511/1014] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:20.908958 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6130afc-550d-488f-b05b-62231d18f0e3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:21.010343 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6130afc-550d-488f-b05b-62231d18f0e3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:21.044274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837537a-908e-4d09-a166-ca9acf67d565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:21.045037 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837537a-908e-4d09-a166-ca9acf67d565 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:21.045418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837537a-908e-4d09-a166-ca9acf67d565 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:21.045792 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837537a-908e-4d09-a166-ca9acf67d565 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:21.046567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837537a-908e-4d09-a166-ca9acf67d565 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:21.046977 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837537a-908e-4d09-a166-ca9acf67d565 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:21.048448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837537a-908e-4d09-a166-ca9acf67d565 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:21.053486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 504/1015] 10.222.0.39 () {56 vars in 1112 bytes} [Mon Aug 8 02:14:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:21.058985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b911f08-43ef-44ea-a1b4-63db2ff71018 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:21.148759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:21.171632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b911f08-43ef-44ea-a1b4-63db2ff71018 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:21.205693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817cb73d-0fbc-41cf-9b95-c3ab545ddd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:21.206438 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cb73d-0fbc-41cf-9b95-c3ab545ddd04 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:21.206721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cb73d-0fbc-41cf-9b95-c3ab545ddd04 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:21.206988 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cb73d-0fbc-41cf-9b95-c3ab545ddd04 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:21.207617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817cb73d-0fbc-41cf-9b95-c3ab545ddd04 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:21.207918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817cb73d-0fbc-41cf-9b95-c3ab545ddd04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:21.209268 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817cb73d-0fbc-41cf-9b95-c3ab545ddd04 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:21.218896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 512/1016] 10.222.0.39 () {56 vars in 1112 bytes} [Mon Aug 8 02:14:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:21.219887 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837537a-908e-4d09-a166-ca9acf67d565 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:21.250354 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:21.322812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837537a-908e-4d09-a166-ca9acf67d565 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:21.355922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68457ddd-2faa-49d1-8ad9-c261d080bfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:21.356520 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68457ddd-2faa-49d1-8ad9-c261d080bfc4 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:21.356792 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68457ddd-2faa-49d1-8ad9-c261d080bfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:21.357056 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68457ddd-2faa-49d1-8ad9-c261d080bfc4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:21.357701 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68457ddd-2faa-49d1-8ad9-c261d080bfc4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:21.359088 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68457ddd-2faa-49d1-8ad9-c261d080bfc4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:21.379932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 505/1017] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:14:21 2022] POST /identity/v3/endpoints => generated 334 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:14:21.791229 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c5b59-3cc5-45b7-b481-0d7169601ba9 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:21.791790 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c5b59-3cc5-45b7-b481-0d7169601ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:21.791790 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c5b59-3cc5-45b7-b481-0d7169601ba9 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:21.793249 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 513/1018] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:21.798142 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1c3d5-bb62-4218-b118-a12f20b949ce None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:21.798469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1c3d5-bb62-4218-b118-a12f20b949ce None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:21.798741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1c3d5-bb62-4218-b118-a12f20b949ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:21.817199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-39b1c3d5-bb62-4218-b118-a12f20b949ce None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:21.818411 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-39b1c3d5-bb62-4218-b118-a12f20b949ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:21.894842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 506/1019] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:21 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:21.917235 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c5b59-3cc5-45b7-b481-0d7169601ba9 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:22.019611 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c5b59-3cc5-45b7-b481-0d7169601ba9 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:22.053672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3433a-6374-460b-b9be-d3773918b5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:22.054199 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3433a-6374-460b-b9be-d3773918b5c8 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:22.054503 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3433a-6374-460b-b9be-d3773918b5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:22.054769 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3433a-6374-460b-b9be-d3773918b5c8 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:22.055423 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3433a-6374-460b-b9be-d3773918b5c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:22.056759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3433a-6374-460b-b9be-d3773918b5c8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:22.075603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 514/1020] 10.222.0.39 () {60 vars in 1141 bytes} [Mon Aug 8 02:14:21 2022] POST /identity/v3/services => generated 242 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:14:22.661576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8837adc-1e60-40a1-b871-320f57ac2460 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:22.662089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8837adc-1e60-40a1-b871-320f57ac2460 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:22.662089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8837adc-1e60-40a1-b871-320f57ac2460 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:22.663512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 507/1021] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:22.668675 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2cabb-ca91-41ff-924a-7c563b0e3e59 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:22.668965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2cabb-ca91-41ff-924a-7c563b0e3e59 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:22.669237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2cabb-ca91-41ff-924a-7c563b0e3e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:22.675902 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:22.689445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8dc2cabb-ca91-41ff-924a-7c563b0e3e59 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:22.691278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8dc2cabb-ca91-41ff-924a-7c563b0e3e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:22.764871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 515/1022] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:22.788953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8837adc-1e60-40a1-b871-320f57ac2460 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:22.804569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:22.893083 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8837adc-1e60-40a1-b871-320f57ac2460 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:22.926818 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a7a96-ac15-4e22-b4e3-1910a1cbc2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:22.927444 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a7a96-ac15-4e22-b4e3-1910a1cbc2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:22.927724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a7a96-ac15-4e22-b4e3-1910a1cbc2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:22.928051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a7a96-ac15-4e22-b4e3-1910a1cbc2a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:22.929189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a7a96-ac15-4e22-b4e3-1910a1cbc2a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:22.930666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a7a96-ac15-4e22-b4e3-1910a1cbc2a8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:22.937163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 508/1023] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:22.938750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2cabb-ca91-41ff-924a-7c563b0e3e59 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:23.039537 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2cabb-ca91-41ff-924a-7c563b0e3e59 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:23.066596 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:23.075882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fa73c-6286-4cb9-b074-ae28e3941a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:23.076511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fa73c-6286-4cb9-b074-ae28e3941a7e None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:23.076784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fa73c-6286-4cb9-b074-ae28e3941a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:23.077048 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fa73c-6286-4cb9-b074-ae28e3941a7e None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:23.077699 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fa73c-6286-4cb9-b074-ae28e3941a7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:23.079135 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fa73c-6286-4cb9-b074-ae28e3941a7e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:23.110282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 516/1024] 10.222.0.39 () {60 vars in 1132 bytes} [Mon Aug 8 02:14:22 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:14:23.487391 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628d377-36f9-442f-8145-244e04cb9ff5 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:23.487705 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628d377-36f9-442f-8145-244e04cb9ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:23.487976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628d377-36f9-442f-8145-244e04cb9ff5 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:23.489470 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 509/1025] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:23.494366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be636d0b-1aa8-4b40-8ec6-2dbc4e6873e3 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:23.494654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be636d0b-1aa8-4b40-8ec6-2dbc4e6873e3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:23.494931 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be636d0b-1aa8-4b40-8ec6-2dbc4e6873e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:23.512970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-be636d0b-1aa8-4b40-8ec6-2dbc4e6873e3 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:23.514156 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-be636d0b-1aa8-4b40-8ec6-2dbc4e6873e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:23.580260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 517/1026] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:23.603996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628d377-36f9-442f-8145-244e04cb9ff5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:23.688133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:23.710153 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628d377-36f9-442f-8145-244e04cb9ff5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:23.750729 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a6e50-ecd4-43a0-94bd-fbade0e76c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:23.751348 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a6e50-ecd4-43a0-94bd-fbade0e76c4e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:23.751622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a6e50-ecd4-43a0-94bd-fbade0e76c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:23.751890 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a6e50-ecd4-43a0-94bd-fbade0e76c4e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:23.761923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a6e50-ecd4-43a0-94bd-fbade0e76c4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:23.766529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b68a6e50-ecd4-43a0-94bd-fbade0e76c4e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:14:23.766529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:23.766529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:23.766529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:23.766529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:23.766529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:23.766529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:23.766529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:23.766529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:23.766529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:23.766529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:23.766529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:23.766529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:23.766529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:23.766529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:23.766529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:23.767448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:23.767448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:23.767448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:23.767448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:23.767448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:23.767448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:23.767448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:23.767448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:23.767448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:23.767448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:23.767448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:23.767448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:23.767448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:23.767448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:23.767448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:23.767448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:23.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:23.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:23.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:23.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:23.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:23.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:23.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:23.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:23.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:23.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:23.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:14:23.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:23.768315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 510/1027] 10.222.0.39 () {56 vars in 1120 bytes} [Mon Aug 8 02:14:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:23.775514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be636d0b-1aa8-4b40-8ec6-2dbc4e6873e3 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:23.874816 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be636d0b-1aa8-4b40-8ec6-2dbc4e6873e3 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:23.889434 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:23.914124 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570e7e0-7c2b-456b-8cab-a8c4c41ffb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:23.914770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570e7e0-7c2b-456b-8cab-a8c4c41ffb17 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:23.915049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570e7e0-7c2b-456b-8cab-a8c4c41ffb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:23.915320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570e7e0-7c2b-456b-8cab-a8c4c41ffb17 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:23.915980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d570e7e0-7c2b-456b-8cab-a8c4c41ffb17 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:23.916286 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d570e7e0-7c2b-456b-8cab-a8c4c41ffb17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:23.917645 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d570e7e0-7c2b-456b-8cab-a8c4c41ffb17 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:23.926082 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 518/1028] 10.222.0.39 () {56 vars in 1115 bytes} [Mon Aug 8 02:14:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:23.931499 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a6e50-ecd4-43a0-94bd-fbade0e76c4e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:24.032480 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a6e50-ecd4-43a0-94bd-fbade0e76c4e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:24.065512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e9fa3-7a8f-493c-a296-f26144188822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:24.066671 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e9fa3-7a8f-493c-a296-f26144188822 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.066950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e9fa3-7a8f-493c-a296-f26144188822 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.067217 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e9fa3-7a8f-493c-a296-f26144188822 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.075404 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:24.078096 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e9fa3-7a8f-493c-a296-f26144188822 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:24.079604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e9fa3-7a8f-493c-a296-f26144188822 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:24.090868 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5b21b-d0a6-48c9-9128-15a58b3db54a None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.091353 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5b21b-d0a6-48c9-9128-15a58b3db54a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.091444 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5b21b-d0a6-48c9-9128-15a58b3db54a None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.092719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 519/1029] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:14:24.096601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-569e9fa3-7a8f-493c-a296-f26144188822 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:24.096601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:24.096601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:24.096601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:24.096601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:24.096601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:24.096601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:24.096601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:24.096601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:24.096601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:24.096601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:24.096601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:24.096601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:24.096601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:24.096601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:24.096601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:24.097746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:24.097746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:24.097746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:24.097746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:24.097746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:24.097746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:24.097746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:24.097746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:24.097746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:24.097746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:24.097746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:24.097746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:24.097746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:24.097746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:24.097746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:24.097746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:24.097746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:24.098890 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:24.098890 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:24.098890 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:24.099100 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f3eb8-97b9-4610-98f5-a21c99e6bf7a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.099100 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f3eb8-97b9-4610-98f5-a21c99e6bf7a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.099100 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f3eb8-97b9-4610-98f5-a21c99e6bf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.100488 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 511/1030] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:24.105858 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e9fa3-7a8f-493c-a296-f26144188822 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:24.117991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-fd9f3eb8-97b9-4610-98f5-a21c99e6bf7a None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:24.119579 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-fd9f3eb8-97b9-4610-98f5-a21c99e6bf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:24.190282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 520/1031] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:14:24.209843 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e9fa3-7a8f-493c-a296-f26144188822 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:24.212877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f3eb8-97b9-4610-98f5-a21c99e6bf7a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:24.244692 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b951d24-96fe-4cfd-87d0-c741e7276111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:24.245321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b951d24-96fe-4cfd-87d0-c741e7276111 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.245581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b951d24-96fe-4cfd-87d0-c741e7276111 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.245851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b951d24-96fe-4cfd-87d0-c741e7276111 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.246520 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b951d24-96fe-4cfd-87d0-c741e7276111 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:24.246832 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b951d24-96fe-4cfd-87d0-c741e7276111 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:24.248174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b951d24-96fe-4cfd-87d0-c741e7276111 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:24.265308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 512/1032] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:24.266391 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b951d24-96fe-4cfd-87d0-c741e7276111 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:24.327226 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:24.328440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f3eb8-97b9-4610-98f5-a21c99e6bf7a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:24.338850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:24.369516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304e166-8253-4750-9276-d8f6db72d3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:24.370142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304e166-8253-4750-9276-d8f6db72d3ef None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.370552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304e166-8253-4750-9276-d8f6db72d3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.371000 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304e166-8253-4750-9276-d8f6db72d3ef None admin] PATH_INFO: `/v3/services/image` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.371404 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304e166-8253-4750-9276-d8f6db72d3ef None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:24.372758 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304e166-8253-4750-9276-d8f6db72d3ef None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:24.380196 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b951d24-96fe-4cfd-87d0-c741e7276111 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:24.382097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-d304e166-8253-4750-9276-d8f6db72d3ef None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:14:24.382097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:24.382097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:14:24.382097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:14:24.382097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:14:24.382097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:14:24.382097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:14:24.382097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:24.382097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:14:24.382097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:24.382097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:24.382097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:24.382097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:24.382097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:24.382097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:24.382097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:24.382097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:24.383392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:24.383392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:24.383392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:24.383392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:24.383392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:24.383392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:24.383392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:14:24.383392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:14:24.383392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:14:24.383392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:14:24.383392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:24.383392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:24.383392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:24.383392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:24.383392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:24.383392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:24.384248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:24.384248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:24.384248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:24.384248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:24.384248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:24.384248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:24.384248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:24.384248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:24.384248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:24.384248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:24.384248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:14:24.384248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:24.384248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:14:24.384248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:24.384248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 521/1033] 10.222.0.39 () {56 vars in 1105 bytes} [Mon Aug 8 02:14:24 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:24.389080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304e166-8253-4750-9276-d8f6db72d3ef None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:24.422141 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8baa4-11c1-4ab0-8264-5fe14485bb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:24.422780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8baa4-11c1-4ab0-8264-5fe14485bb43 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.423060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8baa4-11c1-4ab0-8264-5fe14485bb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.423340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8baa4-11c1-4ab0-8264-5fe14485bb43 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.431281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da8baa4-11c1-4ab0-8264-5fe14485bb43 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:24.432727 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da8baa4-11c1-4ab0-8264-5fe14485bb43 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:24.450513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-4da8baa4-11c1-4ab0-8264-5fe14485bb43 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:14:24.450513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:24.450513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:24.450513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:24.450513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:24.450513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:24.450513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:24.450513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:24.450513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:24.450513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:24.450513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:24.450513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:24.450513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:24.450513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:24.450513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:24.450513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:24.451419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:24.451419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:24.451419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:24.451419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:24.451419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:24.451419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:24.451419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:24.451419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:24.451419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:24.451419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:24.451419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:24.451419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:24.451419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:24.451419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:24.451419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:24.451419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:24.451419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:24.451419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:24.452349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:24.452349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:24.452349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:24.452349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:24.452349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:24.452349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:24.452349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:24.452349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:24.452349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:14:24.452349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:24.452349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 513/1034] 10.222.0.39 () {56 vars in 1093 bytes} [Mon Aug 8 02:14:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:24.452349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8baa4-11c1-4ab0-8264-5fe14485bb43 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:24.513702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304e166-8253-4750-9276-d8f6db72d3ef None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:24.549829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149364ac-5f3d-4f7b-a68b-036789d2789b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:24.551240 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149364ac-5f3d-4f7b-a68b-036789d2789b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.551509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149364ac-5f3d-4f7b-a68b-036789d2789b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.551822 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149364ac-5f3d-4f7b-a68b-036789d2789b None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.552445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149364ac-5f3d-4f7b-a68b-036789d2789b None admin] RBAC: Adding query filter params (name=image) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:24.552741 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149364ac-5f3d-4f7b-a68b-036789d2789b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:24.554083 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149364ac-5f3d-4f7b-a68b-036789d2789b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:24.558838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 522/1035] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:24.559992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8baa4-11c1-4ab0-8264-5fe14485bb43 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:24.564658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149364ac-5f3d-4f7b-a68b-036789d2789b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:24.576760 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:24.601778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b331a-4588-4c9e-a77a-9feb4be695a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:24.601971 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:24.602329 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b331a-4588-4c9e-a77a-9feb4be695a6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.602652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b331a-4588-4c9e-a77a-9feb4be695a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.602923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b331a-4588-4c9e-a77a-9feb4be695a6 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.603640 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b331a-4588-4c9e-a77a-9feb4be695a6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:24.603949 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b331a-4588-4c9e-a77a-9feb4be695a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:24.605394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b331a-4588-4c9e-a77a-9feb4be695a6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:24.628778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 514/1036] 10.222.0.39 () {56 vars in 1133 bytes} [Mon Aug 8 02:14:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:24.630147 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b331a-4588-4c9e-a77a-9feb4be695a6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:24.672524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149364ac-5f3d-4f7b-a68b-036789d2789b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:24.708860 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0a430-8f9c-4316-a4fc-dd3902847f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:24.709461 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0a430-8f9c-4316-a4fc-dd3902847f9d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.709739 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0a430-8f9c-4316-a4fc-dd3902847f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.710012 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0a430-8f9c-4316-a4fc-dd3902847f9d None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.710741 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0a430-8f9c-4316-a4fc-dd3902847f9d None admin] RBAC: Adding query filter params (type=image) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:24.711050 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0a430-8f9c-4316-a4fc-dd3902847f9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:24.712406 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0a430-8f9c-4316-a4fc-dd3902847f9d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:24.717646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 523/1037] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:24.723843 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0a430-8f9c-4316-a4fc-dd3902847f9d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:24.736151 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b331a-4588-4c9e-a77a-9feb4be695a6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:24.772476 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1d7cc-a91f-4a29-a9e3-7df43d10ade7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:24.773075 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1d7cc-a91f-4a29-a9e3-7df43d10ade7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.773350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1d7cc-a91f-4a29-a9e3-7df43d10ade7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.773634 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1d7cc-a91f-4a29-a9e3-7df43d10ade7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.822421 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:24.823199 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:24.828158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0a430-8f9c-4316-a4fc-dd3902847f9d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:24.856019 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1d7cc-a91f-4a29-a9e3-7df43d10ade7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:24.857617 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1d7cc-a91f-4a29-a9e3-7df43d10ade7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:24.863291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf56f93-ffe7-42c9-aeaf-e5de68eddd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:24.863900 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf56f93-ffe7-42c9-aeaf-e5de68eddd4a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.864176 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf56f93-ffe7-42c9-aeaf-e5de68eddd4a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.864445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf56f93-ffe7-42c9-aeaf-e5de68eddd4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.865146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf56f93-ffe7-42c9-aeaf-e5de68eddd4a None admin] RBAC: Adding query filter params (interface=public, service_id=4da77921ad1e41299453515f2d36bcef, region_id=RegionOne) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:24.865452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf56f93-ffe7-42c9-aeaf-e5de68eddd4a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:24.866817 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf56f93-ffe7-42c9-aeaf-e5de68eddd4a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:24.871856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 524/1038] 10.222.0.39 () {56 vars in 1251 bytes} [Mon Aug 8 02:14:24 2022] GET /identity/v3/endpoints?service_id=4da77921ad1e41299453515f2d36bcef&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:24.877327 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf56f93-ffe7-42c9-aeaf-e5de68eddd4a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:24.967658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 515/1039] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:24 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:14:24.968682 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1d7cc-a91f-4a29-a9e3-7df43d10ade7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:24.998955 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf56f93-ffe7-42c9-aeaf-e5de68eddd4a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:25.035654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87feb524-2bb4-4dcb-9d3e-19d01169ca3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.036253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feb524-2bb4-4dcb-9d3e-19d01169ca3f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.036527 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feb524-2bb4-4dcb-9d3e-19d01169ca3f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.036794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feb524-2bb4-4dcb-9d3e-19d01169ca3f None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.037393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87feb524-2bb4-4dcb-9d3e-19d01169ca3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:25.038774 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87feb524-2bb4-4dcb-9d3e-19d01169ca3f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:25.046290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 525/1040] 10.222.0.39 () {56 vars in 1087 bytes} [Mon Aug 8 02:14:24 2022] GET /identity/v3/services => generated 1957 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:14:25.046445 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:25.086416 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1d7cc-a91f-4a29-a9e3-7df43d10ade7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:25.125051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72811c4-aa94-42b2-bc9a-e0911f2c527a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.125608 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72811c4-aa94-42b2-bc9a-e0911f2c527a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.125882 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72811c4-aa94-42b2-bc9a-e0911f2c527a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.126149 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72811c4-aa94-42b2-bc9a-e0911f2c527a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.137582 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72811c4-aa94-42b2-bc9a-e0911f2c527a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:25.139120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72811c4-aa94-42b2-bc9a-e0911f2c527a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:25.155259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a72811c4-aa94-42b2-bc9a-e0911f2c527a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:25.155259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:25.155259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:25.155259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:25.155259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:25.155259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:25.155259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:25.155259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:25.155259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:25.155259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:25.155259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:25.155259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:25.155259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:25.155259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:25.155259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:25.155259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:25.156283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:25.156283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:25.156283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:25.156283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:25.156283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:25.156283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:25.156283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:25.156283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:25.156283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:25.156283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:25.156283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:25.156283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:25.156283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:25.156283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:25.156283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:25.156283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:25.156283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:25.157209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:25.157209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:25.157209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:25.157209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 516/1041] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:25.162964 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87feb524-2bb4-4dcb-9d3e-19d01169ca3f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:25.193128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d201aa0-d78f-45cd-827d-96c2f40ec7e7 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.193565 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d201aa0-d78f-45cd-827d-96c2f40ec7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.193785 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d201aa0-d78f-45cd-827d-96c2f40ec7e7 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.194964 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:25.196471 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 517/1042] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:25.202370 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d2d57-2ace-462b-9c50-e07b477fda7a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.202697 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d2d57-2ace-462b-9c50-e07b477fda7a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.202925 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d2d57-2ace-462b-9c50-e07b477fda7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.227274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-141d2d57-2ace-462b-9c50-e07b477fda7a None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:25.228494 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-141d2d57-2ace-462b-9c50-e07b477fda7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:25.278046 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87feb524-2bb4-4dcb-9d3e-19d01169ca3f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:25.302875 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 518/1043] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:14:25.315110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59026d-b94a-4073-b6f6-b4a4edc8d310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.316238 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59026d-b94a-4073-b6f6-b4a4edc8d310 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.316516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59026d-b94a-4073-b6f6-b4a4edc8d310 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.316788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59026d-b94a-4073-b6f6-b4a4edc8d310 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.317427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd59026d-b94a-4073-b6f6-b4a4edc8d310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:25.317725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd59026d-b94a-4073-b6f6-b4a4edc8d310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:25.319124 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd59026d-b94a-4073-b6f6-b4a4edc8d310 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:25.327256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d2d57-2ace-462b-9c50-e07b477fda7a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:25.332181 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 526/1044] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:25.338716 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59026d-b94a-4073-b6f6-b4a4edc8d310 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:25.382104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:25.436364 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d2d57-2ace-462b-9c50-e07b477fda7a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:25.437897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:25.454493 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59026d-b94a-4073-b6f6-b4a4edc8d310 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:25.476845 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65cc80-df69-4bce-ba70-9e60ee25623e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.477469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65cc80-df69-4bce-ba70-9e60ee25623e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.477739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65cc80-df69-4bce-ba70-9e60ee25623e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.478006 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65cc80-df69-4bce-ba70-9e60ee25623e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.489052 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65cc80-df69-4bce-ba70-9e60ee25623e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:25.492621 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7311e15-d0db-4bca-ae18-914e94078295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.493244 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7311e15-d0db-4bca-ae18-914e94078295 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.493522 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7311e15-d0db-4bca-ae18-914e94078295 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.493796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7311e15-d0db-4bca-ae18-914e94078295 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.494120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1c65cc80-df69-4bce-ba70-9e60ee25623e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:25.494120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:25.494120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:25.494120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:25.494120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:25.494120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:25.494120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:25.494120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:25.494120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:25.494120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:25.494120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:25.494120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:25.494120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:25.494120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:25.494120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:25.494120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:25.495070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:25.495070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:25.495070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:25.495070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:25.495070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:25.495070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:25.495070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:25.495070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:25.495070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:25.495070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:25.495070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:25.495070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:25.495070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:25.495070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:25.495070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:25.495070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:25.496263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:25.496263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:25.496263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:25.496263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:25.496263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:25.496263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:25.496263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:25.496263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:25.496263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:25.496263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:25.496263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:25.496263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:25.496263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 519/1045] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:14:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:25.503205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65cc80-df69-4bce-ba70-9e60ee25623e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:25.503471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7311e15-d0db-4bca-ae18-914e94078295 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:25.505017 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7311e15-d0db-4bca-ae18-914e94078295 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:25.516413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-d7311e15-d0db-4bca-ae18-914e94078295 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:25.516413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:25.516413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:25.516413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:25.516413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:25.516413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:25.516413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:25.516413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:25.516413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:25.516413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:25.516413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:25.516413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:25.516413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:25.516413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:25.516413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:25.517296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:25.517296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:25.517296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:25.517296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:25.517296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:25.517296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:25.517296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:25.517296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:25.517296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:25.517296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:25.517296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:25.517296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:25.517296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:25.517296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:25.517296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:25.517296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:25.517296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:25.518453 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:25.518453 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:25.518453 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:25.518453 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:25.518453 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:25.518453 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:25.518453 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:25.518453 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:25.518453 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 527/1046] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:25.523245 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7311e15-d0db-4bca-ae18-914e94078295 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:25.613078 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65cc80-df69-4bce-ba70-9e60ee25623e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:25.627236 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:25.631644 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7311e15-d0db-4bca-ae18-914e94078295 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:25.650000 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecc85c-0eca-4266-b909-3d812cebeb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.650562 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecc85c-0eca-4266-b909-3d812cebeb2a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.650840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecc85c-0eca-4266-b909-3d812cebeb2a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.651109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecc85c-0eca-4266-b909-3d812cebeb2a None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.651758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ecc85c-0eca-4266-b909-3d812cebeb2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:25.652058 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ecc85c-0eca-4266-b909-3d812cebeb2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:25.653502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ecc85c-0eca-4266-b909-3d812cebeb2a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:25.666738 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 520/1047] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:25.667738 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecc85c-0eca-4266-b909-3d812cebeb2a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:25.672877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440547bc-4e69-4f04-9595-bcbba8e71f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.673477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440547bc-4e69-4f04-9595-bcbba8e71f41 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.673751 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440547bc-4e69-4f04-9595-bcbba8e71f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.674019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440547bc-4e69-4f04-9595-bcbba8e71f41 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.674730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440547bc-4e69-4f04-9595-bcbba8e71f41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:25.675049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440547bc-4e69-4f04-9595-bcbba8e71f41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:25.676510 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440547bc-4e69-4f04-9595-bcbba8e71f41 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:25.690508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 528/1048] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:25.696229 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440547bc-4e69-4f04-9595-bcbba8e71f41 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:25.705706 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:25.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecc85c-0eca-4266-b909-3d812cebeb2a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:25.806636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440547bc-4e69-4f04-9595-bcbba8e71f41 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:25.815343 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf0e0c-9d30-4157-9046-0c344dfdd131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.816008 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf0e0c-9d30-4157-9046-0c344dfdd131 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.816291 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf0e0c-9d30-4157-9046-0c344dfdd131 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.816569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf0e0c-9d30-4157-9046-0c344dfdd131 None admin] PATH_INFO: `/v3/users/3e1596c421f0475cb412300797b32b8e` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.830383 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf0e0c-9d30-4157-9046-0c344dfdd131 None admin] RBAC: Authorizing `identity:get_user(user_id=3e1596c421f0475cb412300797b32b8e)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:25.832453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf0e0c-9d30-4157-9046-0c344dfdd131 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:25.834093 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 521/1049] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:25 2022] GET /identity/v3/users/3e1596c421f0475cb412300797b32b8e => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:25.839672 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf0e0c-9d30-4157-9046-0c344dfdd131 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:25.848521 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336aaece-6747-4a0b-bb12-9b138f5e9fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.849122 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336aaece-6747-4a0b-bb12-9b138f5e9fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.849395 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336aaece-6747-4a0b-bb12-9b138f5e9fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.849661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336aaece-6747-4a0b-bb12-9b138f5e9fc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.850754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336aaece-6747-4a0b-bb12-9b138f5e9fc1 None admin] RBAC: Adding query filter params (role.id=ade71d0856a946a0a91f6097a102aa7c, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=c4238566a9c248ffbfa314f47f02670c) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:25.851077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336aaece-6747-4a0b-bb12-9b138f5e9fc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:25.852782 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336aaece-6747-4a0b-bb12-9b138f5e9fc1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:25.858694 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 529/1050] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:25 2022] GET /identity/v3/role_assignments?user.id=c4238566a9c248ffbfa314f47f02670c&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=ade71d0856a946a0a91f6097a102aa7c => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:25.876814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:25.957203 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf0e0c-9d30-4157-9046-0c344dfdd131 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:25.993744 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60b8ec-dfcd-4f41-aa4b-d99824498fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.994711 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60b8ec-dfcd-4f41-aa4b-d99824498fda None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.995197 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60b8ec-dfcd-4f41-aa4b-d99824498fda None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.995674 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60b8ec-dfcd-4f41-aa4b-d99824498fda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:26.064915 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60b8ec-dfcd-4f41-aa4b-d99824498fda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:26.067224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60b8ec-dfcd-4f41-aa4b-d99824498fda None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:26.098055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:26.170670 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 522/1051] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:25 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 02:14:26.176846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336aaece-6747-4a0b-bb12-9b138f5e9fc1 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:26.249699 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:26.283315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336aaece-6747-4a0b-bb12-9b138f5e9fc1 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:26.317792 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b93d73-05bb-471e-81f6-393cc89dbef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:26.318454 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b93d73-05bb-471e-81f6-393cc89dbef1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:26.318737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b93d73-05bb-471e-81f6-393cc89dbef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:26.319012 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b93d73-05bb-471e-81f6-393cc89dbef1 None admin] PATH_INFO: `/v3/projects/8324e362dedd4b9b9b8a489eeda4eb43` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:26.320127 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b93d73-05bb-471e-81f6-393cc89dbef1 None admin] RBAC: Authorizing `identity:get_project(project_id=8324e362dedd4b9b9b8a489eeda4eb43)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:26.321559 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b93d73-05bb-471e-81f6-393cc89dbef1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:26.323390 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 530/1052] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:26 2022] GET /identity/v3/projects/8324e362dedd4b9b9b8a489eeda4eb43 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:26.329342 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60b8ec-dfcd-4f41-aa4b-d99824498fda None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:26.436252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60b8ec-dfcd-4f41-aa4b-d99824498fda None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:26.468120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:26.474401 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812016f-f936-43f0-bd83-1f789e87d3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:26.475008 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812016f-f936-43f0-bd83-1f789e87d3cd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:26.475412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812016f-f936-43f0-bd83-1f789e87d3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:26.475588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812016f-f936-43f0-bd83-1f789e87d3cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:26.476279 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812016f-f936-43f0-bd83-1f789e87d3cd None admin] RBAC: Adding query filter params (role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.project.id=8324e362dedd4b9b9b8a489eeda4eb43, user.id=3e1596c421f0475cb412300797b32b8e) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:26.476593 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812016f-f936-43f0-bd83-1f789e87d3cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:26.478055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812016f-f936-43f0-bd83-1f789e87d3cd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:26.490274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 523/1053] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:26 2022] GET /identity/v3/role_assignments?user.id=3e1596c421f0475cb412300797b32b8e&scope.project.id=8324e362dedd4b9b9b8a489eeda4eb43&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:27.084909 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d444d-ab67-4dbd-9725-a570b0af0c5c None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:27.085282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d444d-ab67-4dbd-9725-a570b0af0c5c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:27.085494 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d444d-ab67-4dbd-9725-a570b0af0c5c None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:27.086872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 531/1054] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:27.091910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca680d8f-af91-4743-a0f3-6dd7f556f64e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:27.092357 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca680d8f-af91-4743-a0f3-6dd7f556f64e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:27.092434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca680d8f-af91-4743-a0f3-6dd7f556f64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:27.110447 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ca680d8f-af91-4743-a0f3-6dd7f556f64e None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:27.111617 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ca680d8f-af91-4743-a0f3-6dd7f556f64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:27.179594 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 524/1055] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:27.202204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d444d-ab67-4dbd-9725-a570b0af0c5c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:27.301797 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d444d-ab67-4dbd-9725-a570b0af0c5c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:27.336209 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea477a-e007-4d32-8eea-7d0833acce51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:27.336746 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea477a-e007-4d32-8eea-7d0833acce51 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:27.337018 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea477a-e007-4d32-8eea-7d0833acce51 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:27.337282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea477a-e007-4d32-8eea-7d0833acce51 None admin] PATH_INFO: `/v3/services/image` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:27.337851 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea477a-e007-4d32-8eea-7d0833acce51 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:27.339223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea477a-e007-4d32-8eea-7d0833acce51 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:27.347740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-ceea477a-e007-4d32-8eea-7d0833acce51 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:14:27.347740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:27.347740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:14:27.347740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:14:27.347740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:14:27.347740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:14:27.347740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:14:27.347740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:27.347740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:14:27.347740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:27.347740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:27.347740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:27.347740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:27.347740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:27.347740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:27.347740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:27.347740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:27.348721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:27.348721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:27.348721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:27.348721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:27.348721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:27.348721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:27.348721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:14:27.348721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:14:27.348721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:14:27.348721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:14:27.348721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:27.348721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:27.348721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:27.348721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:27.348721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:27.348721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:27.349565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:27.349565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:27.349565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:27.349565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:27.349565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:27.349565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:27.349565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:27.349565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:27.349565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:27.349565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:27.349565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:14:27.349565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:27.349565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:14:27.349565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:27.353823 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 532/1056] 10.222.0.39 () {56 vars in 1105 bytes} [Mon Aug 8 02:14:27 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:27.354665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca680d8f-af91-4743-a0f3-6dd7f556f64e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:27.463318 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca680d8f-af91-4743-a0f3-6dd7f556f64e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:27.488861 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:27.499089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079b2e5-2e26-4695-904a-f16ec93dc90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:27.499692 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079b2e5-2e26-4695-904a-f16ec93dc90d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:27.499966 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079b2e5-2e26-4695-904a-f16ec93dc90d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:27.500230 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079b2e5-2e26-4695-904a-f16ec93dc90d None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:27.500851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9079b2e5-2e26-4695-904a-f16ec93dc90d None admin] RBAC: Adding query filter params (name=image) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:27.501147 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9079b2e5-2e26-4695-904a-f16ec93dc90d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:27.502523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9079b2e5-2e26-4695-904a-f16ec93dc90d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:27.512803 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 525/1057] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:27.514162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea477a-e007-4d32-8eea-7d0833acce51 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:27.517126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:27.622575 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea477a-e007-4d32-8eea-7d0833acce51 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:27.655834 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e543ca-51bd-4725-983d-02b8bbfb15de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:27.656434 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e543ca-51bd-4725-983d-02b8bbfb15de None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:27.656712 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e543ca-51bd-4725-983d-02b8bbfb15de None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:27.656984 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e543ca-51bd-4725-983d-02b8bbfb15de None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:27.657673 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e543ca-51bd-4725-983d-02b8bbfb15de None admin] RBAC: Adding query filter params (type=image) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:27.657979 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e543ca-51bd-4725-983d-02b8bbfb15de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:27.659931 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e543ca-51bd-4725-983d-02b8bbfb15de None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:27.669369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 533/1058] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:27.670746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079b2e5-2e26-4695-904a-f16ec93dc90d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:27.771081 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079b2e5-2e26-4695-904a-f16ec93dc90d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:27.804980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc92aeb-3740-4736-9fae-bfa7822d4d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:27.805587 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc92aeb-3740-4736-9fae-bfa7822d4d5f None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:27.805857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc92aeb-3740-4736-9fae-bfa7822d4d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:27.806184 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc92aeb-3740-4736-9fae-bfa7822d4d5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:27.806874 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc92aeb-3740-4736-9fae-bfa7822d4d5f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:27.808224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc92aeb-3740-4736-9fae-bfa7822d4d5f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:27.827583 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 526/1059] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:14:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:14:27.840746 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0aa82d-95f5-4876-b4d8-a84b2e0b5e10 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:27.841104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0aa82d-95f5-4876-b4d8-a84b2e0b5e10 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:27.841299 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0aa82d-95f5-4876-b4d8-a84b2e0b5e10 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:27.842910 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 534/1060] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:27.849106 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e23aae-7080-4241-b8ac-b708a7bdf826 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:27.849399 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e23aae-7080-4241-b8ac-b708a7bdf826 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:27.849656 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e23aae-7080-4241-b8ac-b708a7bdf826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:27.869477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c1e23aae-7080-4241-b8ac-b708a7bdf826 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:27.870698 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c1e23aae-7080-4241-b8ac-b708a7bdf826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:27.919583 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:27.955691 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 527/1061] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:27.979004 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0aa82d-95f5-4876-b4d8-a84b2e0b5e10 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:28.051632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:28.090954 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0aa82d-95f5-4876-b4d8-a84b2e0b5e10 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:28.125853 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee407d-c49b-4f49-ac2a-34159d9c026c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:28.126504 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee407d-c49b-4f49-ac2a-34159d9c026c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.126783 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee407d-c49b-4f49-ac2a-34159d9c026c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.127052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee407d-c49b-4f49-ac2a-34159d9c026c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.137113 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee407d-c49b-4f49-ac2a-34159d9c026c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:28.141754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-bfee407d-c49b-4f49-ac2a-34159d9c026c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:14:28.141754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:28.141754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:28.141754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:28.141754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:28.141754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:28.141754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:28.141754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:28.141754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:28.141754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:28.141754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:28.141754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:28.141754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:28.141754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:28.141754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:28.141754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:28.142711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:28.142711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:28.142711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:28.142711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:28.142711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:28.142711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:28.142711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:28.142711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:28.142711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:28.142711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:28.142711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:28.142711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:28.142711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:28.142711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:28.142711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:28.142711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:28.144060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:28.144060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:28.144060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:28.144060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:28.144060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:28.144060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:28.144060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:28.144060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:28.144060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:28.144060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:28.144060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:14:28.144060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:28.144060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 535/1062] 10.222.0.39 () {56 vars in 1120 bytes} [Mon Aug 8 02:14:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:28.149414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e23aae-7080-4241-b8ac-b708a7bdf826 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:28.250087 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e23aae-7080-4241-b8ac-b708a7bdf826 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:28.285056 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c83345-41fd-4e63-a178-ff5619d717e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:28.285653 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c83345-41fd-4e63-a178-ff5619d717e4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.285927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c83345-41fd-4e63-a178-ff5619d717e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.286195 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c83345-41fd-4e63-a178-ff5619d717e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.286875 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c83345-41fd-4e63-a178-ff5619d717e4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:28.287234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c83345-41fd-4e63-a178-ff5619d717e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:28.288527 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c83345-41fd-4e63-a178-ff5619d717e4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:28.296699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 528/1063] 10.222.0.39 () {56 vars in 1115 bytes} [Mon Aug 8 02:14:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:28.302616 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee407d-c49b-4f49-ac2a-34159d9c026c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:28.402380 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee407d-c49b-4f49-ac2a-34159d9c026c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:28.429996 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:28.442499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f0e83-1104-49e4-8761-cb7a711d3824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:28.442499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f0e83-1104-49e4-8761-cb7a711d3824 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.442499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f0e83-1104-49e4-8761-cb7a711d3824 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.442499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f0e83-1104-49e4-8761-cb7a711d3824 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.454153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f0e83-1104-49e4-8761-cb7a711d3824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:28.455635 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f0e83-1104-49e4-8761-cb7a711d3824 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:28.469702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-3b1f0e83-1104-49e4-8761-cb7a711d3824 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:28.469702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:28.469702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:28.469702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:28.469702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:28.469702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:28.469702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:28.469702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:28.469702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:28.469702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:28.469702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:28.469702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:28.469702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:28.469702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:28.469702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:28.469702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:28.470854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:28.470854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:28.470854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:28.470854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:28.470854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:28.470854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:28.470854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:28.470854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:28.470854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:28.470854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:28.470854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:28.470854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:28.470854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:28.470854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:28.470854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:28.470854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:28.470854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:28.472257 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:28.472257 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:28.472257 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:28.474563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 536/1064] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:28.477205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c83345-41fd-4e63-a178-ff5619d717e4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:28.482649 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20698cde-b8b8-4970-b168-fa0c875603f0 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.482932 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20698cde-b8b8-4970-b168-fa0c875603f0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.483205 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20698cde-b8b8-4970-b168-fa0c875603f0 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.484713 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 537/1065] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:14:28.490094 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a39b42-cd6d-4d96-ad72-dbb66ac1c28f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.490423 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a39b42-cd6d-4d96-ad72-dbb66ac1c28f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.490696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a39b42-cd6d-4d96-ad72-dbb66ac1c28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.508329 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-77a39b42-cd6d-4d96-ad72-dbb66ac1c28f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:28.509556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-77a39b42-cd6d-4d96-ad72-dbb66ac1c28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:28.513056 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:28.586497 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c83345-41fd-4e63-a178-ff5619d717e4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:28.590388 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 538/1066] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:14:28.611336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a39b42-cd6d-4d96-ad72-dbb66ac1c28f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:28.631864 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b1f92-5191-4e16-8831-c2a95b2a4ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:28.632696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b1f92-5191-4e16-8831-c2a95b2a4ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.633523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b1f92-5191-4e16-8831-c2a95b2a4ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.633954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b1f92-5191-4e16-8831-c2a95b2a4ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.635067 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b1f92-5191-4e16-8831-c2a95b2a4ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:28.635688 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b1f92-5191-4e16-8831-c2a95b2a4ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:28.637799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b1f92-5191-4e16-8831-c2a95b2a4ed4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:28.651129 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 529/1067] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:28.656517 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b1f92-5191-4e16-8831-c2a95b2a4ed4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:28.720742 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a39b42-cd6d-4d96-ad72-dbb66ac1c28f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:28.733812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:28.756993 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:28.759713 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95948f7-6e89-42a7-89f9-7510e6f25424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:28.760577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95948f7-6e89-42a7-89f9-7510e6f25424 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.761168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95948f7-6e89-42a7-89f9-7510e6f25424 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.761351 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95948f7-6e89-42a7-89f9-7510e6f25424 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.764125 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b1f92-5191-4e16-8831-c2a95b2a4ed4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:28.772048 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95948f7-6e89-42a7-89f9-7510e6f25424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:28.776657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-c95948f7-6e89-42a7-89f9-7510e6f25424 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:28.776657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:28.776657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:28.776657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:28.776657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:28.776657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:28.776657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:28.776657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:28.776657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:28.776657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:28.776657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:28.776657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:28.776657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:28.776657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:28.776657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:28.776657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:28.777568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:28.777568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:28.777568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:28.777568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:28.777568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:28.777568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:28.777568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:28.777568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:28.777568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:28.777568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:28.777568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:28.777568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:28.777568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:28.777568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:28.777568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:28.777568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:28.778439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:28.778439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:28.778439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:28.778439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:28.778439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:28.778439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:28.778439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:28.778439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:28.778439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:28.778439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:28.778439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:28.778439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:28.778439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 539/1068] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:14:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:28.783594 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95948f7-6e89-42a7-89f9-7510e6f25424 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:28.804203 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adcc27-6be0-4c07-8cd2-2d9efb333c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:28.804827 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adcc27-6be0-4c07-8cd2-2d9efb333c87 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.805105 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adcc27-6be0-4c07-8cd2-2d9efb333c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.805374 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adcc27-6be0-4c07-8cd2-2d9efb333c87 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.812577 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62adcc27-6be0-4c07-8cd2-2d9efb333c87 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:28.814016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62adcc27-6be0-4c07-8cd2-2d9efb333c87 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:28.825301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-62adcc27-6be0-4c07-8cd2-2d9efb333c87 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:14:28.825301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:28.825301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:28.825301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:28.825301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:28.825301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:28.825301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:28.825301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:28.825301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:28.825301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:28.825301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:28.825301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:28.825301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:28.825301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:28.825301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:28.825301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:28.826283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:28.826283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:28.826283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:28.826283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:28.826283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:28.826283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:28.826283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:28.826283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:28.826283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:28.826283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:28.826283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:28.826283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:28.826283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:28.826283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:28.826283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:28.826283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:28.826283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:28.826283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:28.833017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:28.833017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:28.833017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:28.833017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:28.833017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:28.833017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:28.833017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:28.833017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:28.833017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:14:28.833017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:28.833017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 530/1069] 10.222.0.39 () {56 vars in 1093 bytes} [Mon Aug 8 02:14:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:28.833017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adcc27-6be0-4c07-8cd2-2d9efb333c87 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:28.906275 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95948f7-6e89-42a7-89f9-7510e6f25424 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:28.946154 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede939c7-1a42-4986-abe7-878b7ae07a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:28.946779 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede939c7-1a42-4986-abe7-878b7ae07a33 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.947125 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede939c7-1a42-4986-abe7-878b7ae07a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.947313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede939c7-1a42-4986-abe7-878b7ae07a33 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.947969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede939c7-1a42-4986-abe7-878b7ae07a33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:28.948267 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede939c7-1a42-4986-abe7-878b7ae07a33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:28.949629 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede939c7-1a42-4986-abe7-878b7ae07a33 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:28.955168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adcc27-6be0-4c07-8cd2-2d9efb333c87 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:28.958556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 540/1070] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:28.964284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede939c7-1a42-4986-abe7-878b7ae07a33 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:28.999627 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9743ea3b-bce2-4bb8-9a2b-afd351e91418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.000217 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743ea3b-bce2-4bb8-9a2b-afd351e91418 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.000491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743ea3b-bce2-4bb8-9a2b-afd351e91418 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.000753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743ea3b-bce2-4bb8-9a2b-afd351e91418 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.001471 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9743ea3b-bce2-4bb8-9a2b-afd351e91418 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:29.001770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9743ea3b-bce2-4bb8-9a2b-afd351e91418 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:29.003252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9743ea3b-bce2-4bb8-9a2b-afd351e91418 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:29.008838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:29.023130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 531/1071] 10.222.0.39 () {56 vars in 1133 bytes} [Mon Aug 8 02:14:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:29.028348 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9743ea3b-bce2-4bb8-9a2b-afd351e91418 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:29.044009 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:29.093682 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede939c7-1a42-4986-abe7-878b7ae07a33 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:29.130755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e063076-1530-4882-8eef-77d8983c56cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.131381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e063076-1530-4882-8eef-77d8983c56cb None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.131658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e063076-1530-4882-8eef-77d8983c56cb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.131927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e063076-1530-4882-8eef-77d8983c56cb None admin] PATH_INFO: `/v3/users/3e1596c421f0475cb412300797b32b8e` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.133099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e063076-1530-4882-8eef-77d8983c56cb None admin] RBAC: Authorizing `identity:get_user(user_id=3e1596c421f0475cb412300797b32b8e)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:29.134576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e063076-1530-4882-8eef-77d8983c56cb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:29.136167 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 541/1072] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:28 2022] GET /identity/v3/users/3e1596c421f0475cb412300797b32b8e => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:29.141736 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e063076-1530-4882-8eef-77d8983c56cb None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:29.147046 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9743ea3b-bce2-4bb8-9a2b-afd351e91418 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:29.186560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a1b50-3c2b-4aed-b5de-dbd6eac2c27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.189457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a1b50-3c2b-4aed-b5de-dbd6eac2c27f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.189457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a1b50-3c2b-4aed-b5de-dbd6eac2c27f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.189457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a1b50-3c2b-4aed-b5de-dbd6eac2c27f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.199784 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208a1b50-3c2b-4aed-b5de-dbd6eac2c27f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:29.201225 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208a1b50-3c2b-4aed-b5de-dbd6eac2c27f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:29.216252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-208a1b50-3c2b-4aed-b5de-dbd6eac2c27f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:29.216252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:29.216252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:29.216252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:29.216252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:29.216252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:29.216252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:29.216252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:29.216252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:29.216252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:29.216252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:29.216252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:29.216252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:29.216252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:29.216252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:29.216252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:29.217271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:29.217271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:29.217271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:29.217271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:29.217271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:29.217271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:29.217271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:29.217271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:29.217271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:29.217271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:29.217271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:29.217271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:29.217271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:29.217271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:29.217271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:29.217271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:29.217271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:29.218761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:29.218761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:29.218761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:29.218761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 532/1073] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:29.222948 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a1b50-3c2b-4aed-b5de-dbd6eac2c27f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:29.247508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:29.253463 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e063076-1530-4882-8eef-77d8983c56cb None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:29.285440 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:29.291249 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b60ee-ddcf-4410-a716-4275ede583cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.292029 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b60ee-ddcf-4410-a716-4275ede583cd None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.292282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b60ee-ddcf-4410-a716-4275ede583cd None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.292551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b60ee-ddcf-4410-a716-4275ede583cd None admin] PATH_INFO: `/v3/projects/8324e362dedd4b9b9b8a489eeda4eb43` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.293692 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b60ee-ddcf-4410-a716-4275ede583cd None admin] RBAC: Authorizing `identity:get_project(project_id=8324e362dedd4b9b9b8a489eeda4eb43)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:29.295162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b60ee-ddcf-4410-a716-4275ede583cd None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:29.301557 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 542/1074] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:29 2022] GET /identity/v3/projects/8324e362dedd4b9b9b8a489eeda4eb43 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:29.303333 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b60ee-ddcf-4410-a716-4275ede583cd None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:29.328075 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a1b50-3c2b-4aed-b5de-dbd6eac2c27f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:29.364542 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4042f43-ded1-4d8b-b940-91aee9c151ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.365147 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4042f43-ded1-4d8b-b940-91aee9c151ec None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.365429 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4042f43-ded1-4d8b-b940-91aee9c151ec None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.365701 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4042f43-ded1-4d8b-b940-91aee9c151ec None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.366481 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4042f43-ded1-4d8b-b940-91aee9c151ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:29.367044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4042f43-ded1-4d8b-b940-91aee9c151ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:29.368259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4042f43-ded1-4d8b-b940-91aee9c151ec None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:29.381474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 533/1075] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:29.386792 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4042f43-ded1-4d8b-b940-91aee9c151ec None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:29.412425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b60ee-ddcf-4410-a716-4275ede583cd None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:29.457259 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889a984-4cf7-45ee-a5f8-04f976326be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.458069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889a984-4cf7-45ee-a5f8-04f976326be6 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.458190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889a984-4cf7-45ee-a5f8-04f976326be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.458462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889a984-4cf7-45ee-a5f8-04f976326be6 None admin] PATH_INFO: `/v3/projects/8324e362dedd4b9b9b8a489eeda4eb43/users/3e1596c421f0475cb412300797b32b8e/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.460521 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889a984-4cf7-45ee-a5f8-04f976326be6 None admin] RBAC: Authorizing `identity:create_grant(project_id=8324e362dedd4b9b9b8a489eeda4eb43, user_id=3e1596c421f0475cb412300797b32b8e, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:29.461953 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889a984-4cf7-45ee-a5f8-04f976326be6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:29.476034 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 543/1076] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:14:29 2022] PUT /identity/v3/projects/8324e362dedd4b9b9b8a489eeda4eb43/users/3e1596c421f0475cb412300797b32b8e/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:14:29.507205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4042f43-ded1-4d8b-b940-91aee9c151ec None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:29.535952 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:29.548671 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52156d72-746a-47fa-b888-f60b5d6cfa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.549406 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52156d72-746a-47fa-b888-f60b5d6cfa15 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.549792 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52156d72-746a-47fa-b888-f60b5d6cfa15 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.550130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52156d72-746a-47fa-b888-f60b5d6cfa15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.557801 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52156d72-746a-47fa-b888-f60b5d6cfa15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:29.559418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52156d72-746a-47fa-b888-f60b5d6cfa15 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:29.572057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-52156d72-746a-47fa-b888-f60b5d6cfa15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:29.572057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:29.572057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:29.572057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:29.572057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:29.572057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:29.572057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:29.572057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:29.572057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:29.572057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:29.572057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:29.572057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:29.572057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:29.572057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:29.572057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:29.573214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:29.573214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:29.573214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:29.573214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:29.573214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:29.573214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:29.573214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:29.573214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:29.573214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:29.573214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:29.573214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:29.573214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:29.573214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:29.573214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:29.573214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:29.573214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:29.573214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:29.574507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:29.574507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:29.574507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:29.574507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:29.574507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:29.574507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:29.574507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:29.574507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:29.575216 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 534/1077] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:29.580739 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889a984-4cf7-45ee-a5f8-04f976326be6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:29.612075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:29.695349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889a984-4cf7-45ee-a5f8-04f976326be6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:29.730844 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b3d90-1a0b-4ded-adfc-53e6c95603d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.731445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b3d90-1a0b-4ded-adfc-53e6c95603d4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.731718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b3d90-1a0b-4ded-adfc-53e6c95603d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.731985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b3d90-1a0b-4ded-adfc-53e6c95603d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.732671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b3d90-1a0b-4ded-adfc-53e6c95603d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:29.732974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b3d90-1a0b-4ded-adfc-53e6c95603d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:29.734441 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b3d90-1a0b-4ded-adfc-53e6c95603d4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:29.747210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 544/1078] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:29.753959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52156d72-746a-47fa-b888-f60b5d6cfa15 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:29.856428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52156d72-746a-47fa-b888-f60b5d6cfa15 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:29.880587 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a4082-489d-4d4b-a5cc-b7e55a00160b None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.881090 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a4082-489d-4d4b-a5cc-b7e55a00160b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.881424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a4082-489d-4d4b-a5cc-b7e55a00160b None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.883345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 545/1079] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:14:29.888237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a4803-8026-441e-b02c-ac33a65f1706 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.888527 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a4803-8026-441e-b02c-ac33a65f1706 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.888805 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a4803-8026-441e-b02c-ac33a65f1706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.894192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb148d81-9ce2-4614-9d25-62a3776550d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.894949 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb148d81-9ce2-4614-9d25-62a3776550d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.895284 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb148d81-9ce2-4614-9d25-62a3776550d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.895821 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb148d81-9ce2-4614-9d25-62a3776550d9 None admin] PATH_INFO: `/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/c4238566a9c248ffbfa314f47f02670c/roles/ade71d0856a946a0a91f6097a102aa7c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.898874 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb148d81-9ce2-4614-9d25-62a3776550d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d474b61f1fe4f17aacb7a3665315c79, user_id=c4238566a9c248ffbfa314f47f02670c, role_id=ade71d0856a946a0a91f6097a102aa7c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:29.900364 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb148d81-9ce2-4614-9d25-62a3776550d9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:29.907149 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-768a4803-8026-441e-b02c-ac33a65f1706 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:29.908362 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-768a4803-8026-441e-b02c-ac33a65f1706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:29.914975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 535/1080] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:14:29 2022] PUT /identity/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/c4238566a9c248ffbfa314f47f02670c/roles/ade71d0856a946a0a91f6097a102aa7c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:14:29.989933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 546/1081] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:14:30.012547 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb148d81-9ce2-4614-9d25-62a3776550d9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:30.070446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:30.129190 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb148d81-9ce2-4614-9d25-62a3776550d9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:30.163501 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05ee63-a3b6-4e24-9918-17e9371eb88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:30.164104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05ee63-a3b6-4e24-9918-17e9371eb88f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:30.164436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05ee63-a3b6-4e24-9918-17e9371eb88f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:30.164711 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05ee63-a3b6-4e24-9918-17e9371eb88f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:30.232576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05ee63-a3b6-4e24-9918-17e9371eb88f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:30.234024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05ee63-a3b6-4e24-9918-17e9371eb88f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:30.278224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:30.335093 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 536/1082] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:14:30.341186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a4803-8026-441e-b02c-ac33a65f1706 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:30.359024 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:30.452885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a4803-8026-441e-b02c-ac33a65f1706 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:30.487733 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d38006-6e96-42ce-aa54-96e6a6e3d859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:30.488357 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d38006-6e96-42ce-aa54-96e6a6e3d859 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:30.488635 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d38006-6e96-42ce-aa54-96e6a6e3d859 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:30.488902 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d38006-6e96-42ce-aa54-96e6a6e3d859 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:30.499016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d38006-6e96-42ce-aa54-96e6a6e3d859 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:30.500446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d38006-6e96-42ce-aa54-96e6a6e3d859 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:30.515581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-57d38006-6e96-42ce-aa54-96e6a6e3d859 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:30.515581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:30.515581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:30.515581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:30.515581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:30.515581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:30.515581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:30.515581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:30.515581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:30.515581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:30.515581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:30.515581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:30.515581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:30.515581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:30.515581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:30.515581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:30.516531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:30.516531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:30.516531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:30.516531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:30.516531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:30.516531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:30.516531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:30.516531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:30.516531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:30.516531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:30.516531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:30.516531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:30.516531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:30.516531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:30.516531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:30.516531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:30.516531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:30.517398 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:30.517398 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:30.517398 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:30.518569 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 547/1083] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:30.522761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05ee63-a3b6-4e24-9918-17e9371eb88f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:30.627250 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05ee63-a3b6-4e24-9918-17e9371eb88f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:30.660816 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ffeb1-4c36-4937-ae13-d26d798d01ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:30.661419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ffeb1-4c36-4937-ae13-d26d798d01ed None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:30.661694 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ffeb1-4c36-4937-ae13-d26d798d01ed None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:30.661957 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ffeb1-4c36-4937-ae13-d26d798d01ed None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:30.662619 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ffeb1-4c36-4937-ae13-d26d798d01ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:30.662922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ffeb1-4c36-4937-ae13-d26d798d01ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:30.664279 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ffeb1-4c36-4937-ae13-d26d798d01ed None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:30.681782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 537/1084] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:31.491642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae43f0-3235-4d3f-b537-19d45e80fed9 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:31.492162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae43f0-3235-4d3f-b537-19d45e80fed9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:31.492162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae43f0-3235-4d3f-b537-19d45e80fed9 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:31.493500 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 548/1085] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:31.498843 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ce8d0-32c4-4955-b60b-48cf894976bb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:31.499257 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ce8d0-32c4-4955-b60b-48cf894976bb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:31.499398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ce8d0-32c4-4955-b60b-48cf894976bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:31.506581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:31.517796 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ac3ce8d0-32c4-4955-b60b-48cf894976bb None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:31.519018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ac3ce8d0-32c4-4955-b60b-48cf894976bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:31.584855 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 538/1086] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:31.608149 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae43f0-3235-4d3f-b537-19d45e80fed9 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:31.673927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:31.708382 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae43f0-3235-4d3f-b537-19d45e80fed9 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:31.740690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e780633-bef7-42c3-b860-0e3bba293ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:31.741303 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e780633-bef7-42c3-b860-0e3bba293ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:31.741577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e780633-bef7-42c3-b860-0e3bba293ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:31.741845 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e780633-bef7-42c3-b860-0e3bba293ae4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:31.751551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e780633-bef7-42c3-b860-0e3bba293ae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:31.756120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-2e780633-bef7-42c3-b860-0e3bba293ae4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:31.756120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:31.756120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:31.756120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:31.756120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:31.756120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:31.756120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:31.756120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:31.756120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:31.756120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:31.756120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:31.756120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:31.756120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:31.756120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:31.756120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:31.756120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:31.757237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:31.757237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:31.757237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:31.757237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:31.757237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:31.757237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:31.757237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:31.757237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:31.757237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:31.757237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:31.757237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:31.757237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:31.757237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:31.757237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:31.757237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:31.757237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:31.758097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:31.758097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:31.758097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:31.758097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:31.758097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:31.758097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:31.758097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:31.758097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:31.758097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:31.758097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:31.758097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:31.758097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:31.758097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 549/1087] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:14:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:31.763380 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ce8d0-32c4-4955-b60b-48cf894976bb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:31.860489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ce8d0-32c4-4955-b60b-48cf894976bb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:31.892887 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737f49d-c517-44ae-be63-e5c6c28a71c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:31.893972 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737f49d-c517-44ae-be63-e5c6c28a71c0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:31.894265 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737f49d-c517-44ae-be63-e5c6c28a71c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:31.894546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737f49d-c517-44ae-be63-e5c6c28a71c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:31.895201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737f49d-c517-44ae-be63-e5c6c28a71c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:31.895504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737f49d-c517-44ae-be63-e5c6c28a71c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:31.896870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737f49d-c517-44ae-be63-e5c6c28a71c0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:31.898229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:31.902172 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93267612-d991-45a6-91a8-b864211d7e98 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:31.902499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93267612-d991-45a6-91a8-b864211d7e98 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:31.902777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93267612-d991-45a6-91a8-b864211d7e98 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:31.904115 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 550/1088] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:31.907543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 539/1089] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:31.909078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb13ae-a261-4e1c-bb8f-faef690318b3 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:31.909942 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb13ae-a261-4e1c-bb8f-faef690318b3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:31.910221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb13ae-a261-4e1c-bb8f-faef690318b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:31.913920 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737f49d-c517-44ae-be63-e5c6c28a71c0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:31.927801 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-e6eb13ae-a261-4e1c-bb8f-faef690318b3 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:31.928997 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e6eb13ae-a261-4e1c-bb8f-faef690318b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:31.995119 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 551/1090] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:32.010419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737f49d-c517-44ae-be63-e5c6c28a71c0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:32.018099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb13ae-a261-4e1c-bb8f-faef690318b3 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:32.044337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bd25c-7175-4486-9f77-f292f78efdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:32.044948 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bd25c-7175-4486-9f77-f292f78efdcf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:32.045229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bd25c-7175-4486-9f77-f292f78efdcf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:32.045497 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bd25c-7175-4486-9f77-f292f78efdcf None admin] PATH_INFO: `/v3/users/3e1596c421f0475cb412300797b32b8e` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:32.046639 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4bd25c-7175-4486-9f77-f292f78efdcf None admin] RBAC: Authorizing `identity:get_user(user_id=3e1596c421f0475cb412300797b32b8e)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:32.048068 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4bd25c-7175-4486-9f77-f292f78efdcf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:32.049640 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 540/1091] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:31 2022] GET /identity/v3/users/3e1596c421f0475cb412300797b32b8e => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:32.055337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bd25c-7175-4486-9f77-f292f78efdcf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:32.077386 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:32.116377 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:32.119230 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb13ae-a261-4e1c-bb8f-faef690318b3 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:32.153268 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d8165-ca7d-40ed-8436-54fd028c5800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:32.153884 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d8165-ca7d-40ed-8436-54fd028c5800 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:32.154164 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d8165-ca7d-40ed-8436-54fd028c5800 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:32.154468 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d8165-ca7d-40ed-8436-54fd028c5800 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:32.155321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bd25c-7175-4486-9f77-f292f78efdcf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:32.165085 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d8165-ca7d-40ed-8436-54fd028c5800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:32.169732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-835d8165-ca7d-40ed-8436-54fd028c5800 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:14:32.169732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:32.169732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:32.169732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:32.169732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:32.169732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:32.169732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:32.169732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:32.169732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:32.169732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:32.169732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:32.169732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:32.169732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:32.169732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:32.169732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:32.169732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:32.170804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:32.170804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:32.170804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:32.170804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:32.170804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:32.170804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:32.170804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:32.170804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:32.170804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:32.170804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:32.170804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:32.170804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:32.170804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:32.170804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:32.170804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:32.170804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:32.171632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:32.171632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:32.171632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:32.171632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:32.171632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:32.171632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:32.171632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:32.171632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:32.171632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:32.171632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:32.171632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:14:32.171632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:32.171632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 552/1092] 10.222.0.39 () {56 vars in 1120 bytes} [Mon Aug 8 02:14:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:32.176757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d8165-ca7d-40ed-8436-54fd028c5800 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:32.188483 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab975d15-f719-46cc-bc07-626cb5cafcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:32.189082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab975d15-f719-46cc-bc07-626cb5cafcbe None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:32.189357 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab975d15-f719-46cc-bc07-626cb5cafcbe None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:32.189624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab975d15-f719-46cc-bc07-626cb5cafcbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:32.258256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab975d15-f719-46cc-bc07-626cb5cafcbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:32.259702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab975d15-f719-46cc-bc07-626cb5cafcbe None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:32.274377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d8165-ca7d-40ed-8436-54fd028c5800 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:32.290899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:32.307878 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648017a-b5d8-4d48-a9e3-7f47468d4adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:32.308469 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648017a-b5d8-4d48-a9e3-7f47468d4adc None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:32.308745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648017a-b5d8-4d48-a9e3-7f47468d4adc None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:32.309011 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648017a-b5d8-4d48-a9e3-7f47468d4adc None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:32.309904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648017a-b5d8-4d48-a9e3-7f47468d4adc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:32.310231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648017a-b5d8-4d48-a9e3-7f47468d4adc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:32.311607 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648017a-b5d8-4d48-a9e3-7f47468d4adc None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:32.319802 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 553/1093] 10.222.0.39 () {56 vars in 1115 bytes} [Mon Aug 8 02:14:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:32.319996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:32.325749 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648017a-b5d8-4d48-a9e3-7f47468d4adc None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:32.364056 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 541/1094] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 02:14:32.369857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab975d15-f719-46cc-bc07-626cb5cafcbe None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:32.425398 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648017a-b5d8-4d48-a9e3-7f47468d4adc None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:32.459416 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d70714-464a-4375-80cd-cdca002997bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:32.460029 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d70714-464a-4375-80cd-cdca002997bb None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:32.460305 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d70714-464a-4375-80cd-cdca002997bb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:32.460577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d70714-464a-4375-80cd-cdca002997bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:32.467893 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab975d15-f719-46cc-bc07-626cb5cafcbe None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:32.470951 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d70714-464a-4375-80cd-cdca002997bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:32.472910 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d70714-464a-4375-80cd-cdca002997bb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:32.486658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-87d70714-464a-4375-80cd-cdca002997bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:32.486658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:32.486658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:32.486658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:32.486658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:32.486658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:32.486658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:32.486658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:32.486658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:32.486658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:32.486658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:32.486658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:32.486658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:32.486658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:32.486658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:32.486658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:32.487627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:32.487627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:32.487627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:32.487627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:32.487627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:32.487627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:32.487627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:32.487627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:32.487627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:32.487627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:32.487627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:32.487627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:32.487627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:32.487627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:32.487627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:32.487627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:32.487627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:32.488502 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:32.488502 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:32.488502 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:32.488502 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 554/1095] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:32.493421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d70714-464a-4375-80cd-cdca002997bb None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:32.502242 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a31a8-2e48-448a-803e-751eb0e186da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:32.502885 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a31a8-2e48-448a-803e-751eb0e186da None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:32.503165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a31a8-2e48-448a-803e-751eb0e186da None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:32.503438 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a31a8-2e48-448a-803e-751eb0e186da None admin] PATH_INFO: `/v3/projects/8324e362dedd4b9b9b8a489eeda4eb43` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:32.504549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a31a8-2e48-448a-803e-751eb0e186da None admin] RBAC: Authorizing `identity:get_project(project_id=8324e362dedd4b9b9b8a489eeda4eb43)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:32.506018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a31a8-2e48-448a-803e-751eb0e186da None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:32.512630 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 542/1096] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:32 2022] GET /identity/v3/projects/8324e362dedd4b9b9b8a489eeda4eb43 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:32.513475 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a31a8-2e48-448a-803e-751eb0e186da None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:32.524346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:32.544412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:32.592881 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d70714-464a-4375-80cd-cdca002997bb None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:32.613233 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a31a8-2e48-448a-803e-751eb0e186da None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:32.626167 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10edb5-cf2b-485a-acc2-854dbac6b0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:32.626788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10edb5-cf2b-485a-acc2-854dbac6b0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:32.627059 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10edb5-cf2b-485a-acc2-854dbac6b0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:32.627324 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10edb5-cf2b-485a-acc2-854dbac6b0f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:32.627980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10edb5-cf2b-485a-acc2-854dbac6b0f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:32.628279 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10edb5-cf2b-485a-acc2-854dbac6b0f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:32.629637 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10edb5-cf2b-485a-acc2-854dbac6b0f3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:32.640840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 555/1097] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:32.646481 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996744e-147d-4d49-99ca-ab6087e735a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:32.646849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10edb5-cf2b-485a-acc2-854dbac6b0f3 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:32.647072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996744e-147d-4d49-99ca-ab6087e735a4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:32.647347 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996744e-147d-4d49-99ca-ab6087e735a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:32.647615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996744e-147d-4d49-99ca-ab6087e735a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:32.648324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996744e-147d-4d49-99ca-ab6087e735a4 None admin] RBAC: Adding query filter params (role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.project.id=8324e362dedd4b9b9b8a489eeda4eb43, user.id=3e1596c421f0475cb412300797b32b8e) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:32.648628 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996744e-147d-4d49-99ca-ab6087e735a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:32.650037 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996744e-147d-4d49-99ca-ab6087e735a4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:32.655674 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 543/1098] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:32 2022] GET /identity/v3/role_assignments?user.id=3e1596c421f0475cb412300797b32b8e&scope.project.id=8324e362dedd4b9b9b8a489eeda4eb43&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:32.746464 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:32.756153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10edb5-cf2b-485a-acc2-854dbac6b0f3 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:32.793431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e36211-9f4d-48c5-bf38-5f0c6e8d42da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:32.793976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e36211-9f4d-48c5-bf38-5f0c6e8d42da None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:32.794280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e36211-9f4d-48c5-bf38-5f0c6e8d42da None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:32.794544 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e36211-9f4d-48c5-bf38-5f0c6e8d42da None admin] PATH_INFO: `/v3/users/nova` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:32.801986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e36211-9f4d-48c5-bf38-5f0c6e8d42da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:32.803466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e36211-9f4d-48c5-bf38-5f0c6e8d42da None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:32.814940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-c5e36211-9f4d-48c5-bf38-5f0c6e8d42da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:14:32.814940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:32.814940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:32.814940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:32.814940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:32.814940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:32.814940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:32.814940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:32.814940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:32.814940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:32.814940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:32.814940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:32.814940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:32.814940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:32.814940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:32.814940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:32.816567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:32.816567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:32.816567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:32.816567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:32.816567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:32.816567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:32.816567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:32.816567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:32.816567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:32.816567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:32.816567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:32.816567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:32.816567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:32.816567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:32.816567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:32.816567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:32.816567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:32.816567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:32.817715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:32.817715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:32.817715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:32.817715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:32.817715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:32.817715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:32.817715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:32.817715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:32.817715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:14:32.817715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:32.817715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 556/1099] 10.222.0.39 () {56 vars in 1093 bytes} [Mon Aug 8 02:14:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:32.822567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996744e-147d-4d49-99ca-ab6087e735a4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:32.916270 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:32.922539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996744e-147d-4d49-99ca-ab6087e735a4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:32.955426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e788377-b855-46a4-b2f6-8c04f63887e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:32.956648 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e788377-b855-46a4-b2f6-8c04f63887e8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:32.956927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e788377-b855-46a4-b2f6-8c04f63887e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:32.957193 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e788377-b855-46a4-b2f6-8c04f63887e8 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:32.957915 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e788377-b855-46a4-b2f6-8c04f63887e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:32.958220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e788377-b855-46a4-b2f6-8c04f63887e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:32.959666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e788377-b855-46a4-b2f6-8c04f63887e8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:32.976083 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 544/1100] 10.222.0.39 () {56 vars in 1133 bytes} [Mon Aug 8 02:14:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:32.981747 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e36211-9f4d-48c5-bf38-5f0c6e8d42da None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:33.078974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e36211-9f4d-48c5-bf38-5f0c6e8d42da None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:33.113493 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a6dc6-f53a-47f2-8b65-40896edb7ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:33.114110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a6dc6-f53a-47f2-8b65-40896edb7ecf None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:33.114416 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a6dc6-f53a-47f2-8b65-40896edb7ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:33.114691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a6dc6-f53a-47f2-8b65-40896edb7ecf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:33.135967 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:33.189512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a6dc6-f53a-47f2-8b65-40896edb7ecf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:33.191836 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a6dc6-f53a-47f2-8b65-40896edb7ecf None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:33.286515 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 557/1101] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:14:33.292214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e788377-b855-46a4-b2f6-8c04f63887e8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:33.389670 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e788377-b855-46a4-b2f6-8c04f63887e8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:33.422285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc4453-61bc-47c8-96e7-2bee99c9011a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:33.422907 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc4453-61bc-47c8-96e7-2bee99c9011a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:33.423185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc4453-61bc-47c8-96e7-2bee99c9011a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:33.423454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc4453-61bc-47c8-96e7-2bee99c9011a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:33.433716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc4453-61bc-47c8-96e7-2bee99c9011a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:33.435450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc4453-61bc-47c8-96e7-2bee99c9011a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:33.452033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-73fc4453-61bc-47c8-96e7-2bee99c9011a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:33.452033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:33.452033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:33.452033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:33.452033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:33.452033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:33.452033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:33.452033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:33.452033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:33.452033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:33.452033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:33.452033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:33.452033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:33.452033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:33.452033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:33.452033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:33.453099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:33.453099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:33.453099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:33.453099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:33.453099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:33.453099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:33.453099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:33.453099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:33.453099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:33.453099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:33.453099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:33.453099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:33.453099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:33.453099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:33.453099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:33.453099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:33.453099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:33.454430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:33.454430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:33.454430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:33.454430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 545/1102] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:33.458939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a6dc6-f53a-47f2-8b65-40896edb7ecf None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:33.509823 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:33.555865 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a6dc6-f53a-47f2-8b65-40896edb7ecf None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:33.588287 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3529fd8-5096-4393-8464-210244a96076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:33.588879 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3529fd8-5096-4393-8464-210244a96076 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:33.589158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3529fd8-5096-4393-8464-210244a96076 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:33.589430 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3529fd8-5096-4393-8464-210244a96076 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:33.590066 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3529fd8-5096-4393-8464-210244a96076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:33.590400 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3529fd8-5096-4393-8464-210244a96076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:33.591761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3529fd8-5096-4393-8464-210244a96076 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:33.603098 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 558/1103] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:33.608553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc4453-61bc-47c8-96e7-2bee99c9011a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:33.618200 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:33.706633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc4453-61bc-47c8-96e7-2bee99c9011a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:33.739815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239cb8f-5380-406e-a688-d31d36fca607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:33.740438 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239cb8f-5380-406e-a688-d31d36fca607 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:33.740715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239cb8f-5380-406e-a688-d31d36fca607 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:33.740985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239cb8f-5380-406e-a688-d31d36fca607 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:33.747582 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239cb8f-5380-406e-a688-d31d36fca607 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:33.749024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239cb8f-5380-406e-a688-d31d36fca607 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:33.759850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-9239cb8f-5380-406e-a688-d31d36fca607 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:33.759850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:33.759850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:33.759850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:33.759850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:33.759850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:33.759850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:33.759850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:33.759850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:33.759850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:33.759850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:33.759850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:33.759850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:33.759850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:33.759850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:33.760889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:33.760889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:33.760889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:33.760889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:33.760889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:33.760889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:33.760889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:33.760889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:33.760889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:33.760889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:33.760889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:33.760889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:33.760889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:33.760889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:33.760889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:33.760889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:33.760889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:33.761944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:33.761944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:33.761944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:33.761944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:33.761944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:33.761944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:33.761944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:33.761944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:33.761944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 546/1104] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:33.766759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3529fd8-5096-4393-8464-210244a96076 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:33.866991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3529fd8-5096-4393-8464-210244a96076 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:33.879499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:33.901332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80309ea-2beb-4124-9d51-58866acace03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:33.901926 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80309ea-2beb-4124-9d51-58866acace03 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:33.902202 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80309ea-2beb-4124-9d51-58866acace03 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:33.902497 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80309ea-2beb-4124-9d51-58866acace03 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:33.903177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80309ea-2beb-4124-9d51-58866acace03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:33.903481 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80309ea-2beb-4124-9d51-58866acace03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:33.904908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80309ea-2beb-4124-9d51-58866acace03 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:33.917123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 559/1105] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:33.923216 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239cb8f-5380-406e-a688-d31d36fca607 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:33.988891 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:34.022889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239cb8f-5380-406e-a688-d31d36fca607 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:34.054942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f2a2d-2bb2-4b0d-808c-f1d72aed4a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:34.055524 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f2a2d-2bb2-4b0d-808c-f1d72aed4a0e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:34.055797 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f2a2d-2bb2-4b0d-808c-f1d72aed4a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:34.056061 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f2a2d-2bb2-4b0d-808c-f1d72aed4a0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:34.056764 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f2a2d-2bb2-4b0d-808c-f1d72aed4a0e None admin] RBAC: Adding query filter params (role.id=ade71d0856a946a0a91f6097a102aa7c, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=c4238566a9c248ffbfa314f47f02670c) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:34.057069 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f2a2d-2bb2-4b0d-808c-f1d72aed4a0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:34.058506 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f2a2d-2bb2-4b0d-808c-f1d72aed4a0e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:34.064234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 547/1106] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:33 2022] GET /identity/v3/role_assignments?user.id=c4238566a9c248ffbfa314f47f02670c&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=ade71d0856a946a0a91f6097a102aa7c => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:34.675624 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca28fc-21da-4374-91ff-69cb6e1238bd None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:34.676193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca28fc-21da-4374-91ff-69cb6e1238bd None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:34.676193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca28fc-21da-4374-91ff-69cb6e1238bd None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:34.677491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 560/1107] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:34.682940 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02d2d0-9e9d-4c1e-8783-390a582fc3a2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:34.683355 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02d2d0-9e9d-4c1e-8783-390a582fc3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:34.683520 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02d2d0-9e9d-4c1e-8783-390a582fc3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:34.702494 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bb02d2d0-9e9d-4c1e-8783-390a582fc3a2 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:34.703614 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bb02d2d0-9e9d-4c1e-8783-390a582fc3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:34.770055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 548/1108] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:14:34.793916 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca28fc-21da-4374-91ff-69cb6e1238bd None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:34.898634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca28fc-21da-4374-91ff-69cb6e1238bd None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:34.932370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4a35e-533f-4e28-946c-2135da9ac1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:34.932987 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4a35e-533f-4e28-946c-2135da9ac1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:34.933266 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4a35e-533f-4e28-946c-2135da9ac1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:34.933535 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4a35e-533f-4e28-946c-2135da9ac1b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:34.934949 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af4a35e-533f-4e28-946c-2135da9ac1b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:34.936385 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af4a35e-533f-4e28-946c-2135da9ac1b1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:34.937964 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 561/1109] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:34.945185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02d2d0-9e9d-4c1e-8783-390a582fc3a2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:35.000572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:35.039891 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02d2d0-9e9d-4c1e-8783-390a582fc3a2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:35.072159 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c0f1e-2268-4ae4-9ca5-ac76a8ada9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:35.072740 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c0f1e-2268-4ae4-9ca5-ac76a8ada9df None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:35.073012 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c0f1e-2268-4ae4-9ca5-ac76a8ada9df None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:35.073276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c0f1e-2268-4ae4-9ca5-ac76a8ada9df None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:35.073912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c0f1e-2268-4ae4-9ca5-ac76a8ada9df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:35.075546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c0f1e-2268-4ae4-9ca5-ac76a8ada9df None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:35.096287 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 549/1110] 10.222.0.39 () {60 vars in 1132 bytes} [Mon Aug 8 02:14:34 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:14:37.118361 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d25f9-1dcd-4fed-98d7-42903a5cf36f None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.118920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d25f9-1dcd-4fed-98d7-42903a5cf36f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.118920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d25f9-1dcd-4fed-98d7-42903a5cf36f None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.120233 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 562/1111] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:37.125278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c52e6-d5fb-40fc-8b84-c04b3594b030 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.125654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c52e6-d5fb-40fc-8b84-c04b3594b030 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.125816 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c52e6-d5fb-40fc-8b84-c04b3594b030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.143191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-725c52e6-d5fb-40fc-8b84-c04b3594b030 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:37.144841 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-725c52e6-d5fb-40fc-8b84-c04b3594b030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:37.208917 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 550/1112] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:37.231801 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d25f9-1dcd-4fed-98d7-42903a5cf36f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:37.272864 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:37.330017 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d25f9-1dcd-4fed-98d7-42903a5cf36f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:37.362528 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4049550-ab3f-4b13-8a45-c74bd7cfd15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:37.363124 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4049550-ab3f-4b13-8a45-c74bd7cfd15b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.363396 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4049550-ab3f-4b13-8a45-c74bd7cfd15b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.363662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4049550-ab3f-4b13-8a45-c74bd7cfd15b None admin] PATH_INFO: `/v3/roles/9f8a5b177f4a41fc8bc88ff4f1ce0839` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.365010 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4049550-ab3f-4b13-8a45-c74bd7cfd15b None admin] RBAC: Authorizing `identity:get_role(role_id=9f8a5b177f4a41fc8bc88ff4f1ce0839)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:37.366383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4049550-ab3f-4b13-8a45-c74bd7cfd15b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:37.367673 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 563/1113] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:37 2022] GET /identity/v3/roles/9f8a5b177f4a41fc8bc88ff4f1ce0839 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:37.373484 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c52e6-d5fb-40fc-8b84-c04b3594b030 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:37.417983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:37.470885 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c52e6-d5fb-40fc-8b84-c04b3594b030 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:37.503215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20b102-1f94-47bb-9847-5a4692ed34ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:37.503827 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20b102-1f94-47bb-9847-5a4692ed34ec None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.504103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20b102-1f94-47bb-9847-5a4692ed34ec None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.504373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20b102-1f94-47bb-9847-5a4692ed34ec None admin] PATH_INFO: `/v3/users/92e5ef8da9f7452fa0153c9f5e46bd3a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.516752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20b102-1f94-47bb-9847-5a4692ed34ec None admin] RBAC: Authorizing `identity:get_user(user_id=92e5ef8da9f7452fa0153c9f5e46bd3a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:37.518191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20b102-1f94-47bb-9847-5a4692ed34ec None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:37.520346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 551/1114] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:37 2022] GET /identity/v3/users/92e5ef8da9f7452fa0153c9f5e46bd3a => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:37.526033 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4049550-ab3f-4b13-8a45-c74bd7cfd15b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:37.621422 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4049550-ab3f-4b13-8a45-c74bd7cfd15b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:37.634455 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:37.660203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243c34f-94d7-4a56-873d-726d4cc0dffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:37.660789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243c34f-94d7-4a56-873d-726d4cc0dffd None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.661062 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243c34f-94d7-4a56-873d-726d4cc0dffd None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.661326 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243c34f-94d7-4a56-873d-726d4cc0dffd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.725051 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243c34f-94d7-4a56-873d-726d4cc0dffd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:37.726491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243c34f-94d7-4a56-873d-726d4cc0dffd None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:37.820669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 564/1115] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:14:37.826746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20b102-1f94-47bb-9847-5a4692ed34ec None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:37.922609 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20b102-1f94-47bb-9847-5a4692ed34ec None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:37.925287 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:37.957163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2f7df-de73-4b3a-8849-49689bfa3943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:37.957774 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2f7df-de73-4b3a-8849-49689bfa3943 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.958053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2f7df-de73-4b3a-8849-49689bfa3943 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.958351 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2f7df-de73-4b3a-8849-49689bfa3943 None admin] PATH_INFO: `/v3/projects/8324e362dedd4b9b9b8a489eeda4eb43` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.959465 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2f7df-de73-4b3a-8849-49689bfa3943 None admin] RBAC: Authorizing `identity:get_project(project_id=8324e362dedd4b9b9b8a489eeda4eb43)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:37.960891 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2f7df-de73-4b3a-8849-49689bfa3943 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:37.962552 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 552/1116] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:37 2022] GET /identity/v3/projects/8324e362dedd4b9b9b8a489eeda4eb43 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:37.968643 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243c34f-94d7-4a56-873d-726d4cc0dffd None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:37.978529 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:38.067473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243c34f-94d7-4a56-873d-726d4cc0dffd None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:38.101879 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e505ff-a157-40a2-8a20-d387fc7f07ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:38.102520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e505ff-a157-40a2-8a20-d387fc7f07ef None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:38.102795 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e505ff-a157-40a2-8a20-d387fc7f07ef None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:38.103061 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e505ff-a157-40a2-8a20-d387fc7f07ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:38.103796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e505ff-a157-40a2-8a20-d387fc7f07ef None admin] RBAC: Adding query filter params (role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839, scope.project.id=8324e362dedd4b9b9b8a489eeda4eb43, user.id=92e5ef8da9f7452fa0153c9f5e46bd3a) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:38.104098 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e505ff-a157-40a2-8a20-d387fc7f07ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:38.105512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e505ff-a157-40a2-8a20-d387fc7f07ef None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:38.110699 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 565/1117] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:37 2022] GET /identity/v3/role_assignments?user.id=92e5ef8da9f7452fa0153c9f5e46bd3a&scope.project.id=8324e362dedd4b9b9b8a489eeda4eb43&role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:40.062765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52e0ed-abb6-41db-b10d-43ba5796e248 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:40.063290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52e0ed-abb6-41db-b10d-43ba5796e248 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:40.063290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52e0ed-abb6-41db-b10d-43ba5796e248 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:40.064622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 553/1118] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:40.069818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec4efb-1fa0-44f8-b5cd-82f197b128e5 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:40.070274 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec4efb-1fa0-44f8-b5cd-82f197b128e5 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:40.070396 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec4efb-1fa0-44f8-b5cd-82f197b128e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:40.088471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d8ec4efb-1fa0-44f8-b5cd-82f197b128e5 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:40.089660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d8ec4efb-1fa0-44f8-b5cd-82f197b128e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:40.154210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 566/1119] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:40.176418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52e0ed-abb6-41db-b10d-43ba5796e248 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:40.274667 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52e0ed-abb6-41db-b10d-43ba5796e248 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:40.309502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad8559-3de0-4152-a725-b4a3405a6764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:40.310396 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad8559-3de0-4152-a725-b4a3405a6764 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:40.310669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad8559-3de0-4152-a725-b4a3405a6764 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:40.310932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad8559-3de0-4152-a725-b4a3405a6764 None admin] PATH_INFO: `/v3/roles/9f8a5b177f4a41fc8bc88ff4f1ce0839` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:40.312293 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad8559-3de0-4152-a725-b4a3405a6764 None admin] RBAC: Authorizing `identity:get_role(role_id=9f8a5b177f4a41fc8bc88ff4f1ce0839)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:40.313703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad8559-3de0-4152-a725-b4a3405a6764 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:40.315004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 554/1120] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:40 2022] GET /identity/v3/roles/9f8a5b177f4a41fc8bc88ff4f1ce0839 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:40.321971 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec4efb-1fa0-44f8-b5cd-82f197b128e5 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:40.324672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:40.419595 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec4efb-1fa0-44f8-b5cd-82f197b128e5 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:40.452472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c722bb-4abc-447c-b294-f09e9b722139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:40.453084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c722bb-4abc-447c-b294-f09e9b722139 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:40.453359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c722bb-4abc-447c-b294-f09e9b722139 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:40.453626 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c722bb-4abc-447c-b294-f09e9b722139 None admin] PATH_INFO: `/v3/users/92e5ef8da9f7452fa0153c9f5e46bd3a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:40.454752 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c722bb-4abc-447c-b294-f09e9b722139 None admin] RBAC: Authorizing `identity:get_user(user_id=92e5ef8da9f7452fa0153c9f5e46bd3a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:40.456847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c722bb-4abc-447c-b294-f09e9b722139 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:40.458455 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 567/1121] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:40 2022] GET /identity/v3/users/92e5ef8da9f7452fa0153c9f5e46bd3a => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:40.464297 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad8559-3de0-4152-a725-b4a3405a6764 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:40.505000 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:40.560997 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad8559-3de0-4152-a725-b4a3405a6764 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:40.595102 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f74d4-058a-446f-8c27-445c048ab638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:40.595653 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f74d4-058a-446f-8c27-445c048ab638 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:40.595928 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f74d4-058a-446f-8c27-445c048ab638 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:40.596196 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f74d4-058a-446f-8c27-445c048ab638 None admin] PATH_INFO: `/v3/projects/8324e362dedd4b9b9b8a489eeda4eb43` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:40.597340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f74d4-058a-446f-8c27-445c048ab638 None admin] RBAC: Authorizing `identity:get_project(project_id=8324e362dedd4b9b9b8a489eeda4eb43)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:40.598805 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f74d4-058a-446f-8c27-445c048ab638 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:40.600424 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 555/1122] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:40 2022] GET /identity/v3/projects/8324e362dedd4b9b9b8a489eeda4eb43 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:40.606560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c722bb-4abc-447c-b294-f09e9b722139 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:40.672850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:40.703774 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c722bb-4abc-447c-b294-f09e9b722139 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:40.735618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e49dbf-9987-4286-bc26-e2d6673d1c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:40.736253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e49dbf-9987-4286-bc26-e2d6673d1c6e None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:40.736530 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e49dbf-9987-4286-bc26-e2d6673d1c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:40.736803 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e49dbf-9987-4286-bc26-e2d6673d1c6e None admin] PATH_INFO: `/v3/projects/8324e362dedd4b9b9b8a489eeda4eb43/users/92e5ef8da9f7452fa0153c9f5e46bd3a/roles/9f8a5b177f4a41fc8bc88ff4f1ce0839` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:40.739450 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e49dbf-9987-4286-bc26-e2d6673d1c6e None admin] RBAC: Authorizing `identity:create_grant(project_id=8324e362dedd4b9b9b8a489eeda4eb43, user_id=92e5ef8da9f7452fa0153c9f5e46bd3a, role_id=9f8a5b177f4a41fc8bc88ff4f1ce0839)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:40.740855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e49dbf-9987-4286-bc26-e2d6673d1c6e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:40.752483 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 568/1123] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:14:40 2022] PUT /identity/v3/projects/8324e362dedd4b9b9b8a489eeda4eb43/users/92e5ef8da9f7452fa0153c9f5e46bd3a/roles/9f8a5b177f4a41fc8bc88ff4f1ce0839 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:14:42.689715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25042a-33fa-4a7f-82c8-65721bc65a41 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:42.690223 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25042a-33fa-4a7f-82c8-65721bc65a41 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:42.690223 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25042a-33fa-4a7f-82c8-65721bc65a41 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:42.691608 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 556/1124] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:42.696630 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee044fa7-4c65-42c8-b8d4-b74b72c4db13 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:42.697003 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee044fa7-4c65-42c8-b8d4-b74b72c4db13 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:42.697100 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee044fa7-4c65-42c8-b8d4-b74b72c4db13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:42.714515 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ee044fa7-4c65-42c8-b8d4-b74b72c4db13 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:42.715674 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ee044fa7-4c65-42c8-b8d4-b74b72c4db13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:42.778607 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 569/1125] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:42.801985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25042a-33fa-4a7f-82c8-65721bc65a41 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:42.900032 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25042a-33fa-4a7f-82c8-65721bc65a41 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:42.909441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:42.933909 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f73395-020a-4f3c-8c95-60f85ccef5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:42.934546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f73395-020a-4f3c-8c95-60f85ccef5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:42.935079 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f73395-020a-4f3c-8c95-60f85ccef5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:42.935352 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f73395-020a-4f3c-8c95-60f85ccef5f4 None admin] PATH_INFO: `/v3/roles/9f8a5b177f4a41fc8bc88ff4f1ce0839` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:42.936731 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f73395-020a-4f3c-8c95-60f85ccef5f4 None admin] RBAC: Authorizing `identity:get_role(role_id=9f8a5b177f4a41fc8bc88ff4f1ce0839)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:42.938087 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f73395-020a-4f3c-8c95-60f85ccef5f4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:42.939489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 557/1126] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:42 2022] GET /identity/v3/roles/9f8a5b177f4a41fc8bc88ff4f1ce0839 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:42.945165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee044fa7-4c65-42c8-b8d4-b74b72c4db13 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:43.002349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:43.044912 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee044fa7-4c65-42c8-b8d4-b74b72c4db13 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:43.077906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84d1fc-d3de-4933-b47f-d529d602fa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:43.078551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84d1fc-d3de-4933-b47f-d529d602fa38 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:43.078834 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84d1fc-d3de-4933-b47f-d529d602fa38 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:43.079109 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84d1fc-d3de-4933-b47f-d529d602fa38 None admin] PATH_INFO: `/v3/users/92e5ef8da9f7452fa0153c9f5e46bd3a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:43.080221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84d1fc-d3de-4933-b47f-d529d602fa38 None admin] RBAC: Authorizing `identity:get_user(user_id=92e5ef8da9f7452fa0153c9f5e46bd3a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:43.082158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84d1fc-d3de-4933-b47f-d529d602fa38 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:43.083766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 570/1127] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:42 2022] GET /identity/v3/users/92e5ef8da9f7452fa0153c9f5e46bd3a => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:43.089734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f73395-020a-4f3c-8c95-60f85ccef5f4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:43.186200 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f73395-020a-4f3c-8c95-60f85ccef5f4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:43.219115 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103c094-157e-4e3f-9b79-82459c6b0c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:43.219712 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103c094-157e-4e3f-9b79-82459c6b0c0a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:43.219992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103c094-157e-4e3f-9b79-82459c6b0c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:43.220262 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103c094-157e-4e3f-9b79-82459c6b0c0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:43.262943 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:43.286307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103c094-157e-4e3f-9b79-82459c6b0c0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:43.287769 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103c094-157e-4e3f-9b79-82459c6b0c0a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:43.382815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 558/1128] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:14:43.388737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84d1fc-d3de-4933-b47f-d529d602fa38 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:43.484528 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84d1fc-d3de-4933-b47f-d529d602fa38 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:43.509520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:43.517516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e715ebb-420b-45da-bf72-ce231f67b56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:43.518070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e715ebb-420b-45da-bf72-ce231f67b56d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:43.518973 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e715ebb-420b-45da-bf72-ce231f67b56d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:43.519247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e715ebb-420b-45da-bf72-ce231f67b56d None admin] PATH_INFO: `/v3/projects/8324e362dedd4b9b9b8a489eeda4eb43` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:43.520358 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e715ebb-420b-45da-bf72-ce231f67b56d None admin] RBAC: Authorizing `identity:get_project(project_id=8324e362dedd4b9b9b8a489eeda4eb43)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:43.521771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e715ebb-420b-45da-bf72-ce231f67b56d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:43.523439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 571/1129] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:43 2022] GET /identity/v3/projects/8324e362dedd4b9b9b8a489eeda4eb43 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:43.529426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103c094-157e-4e3f-9b79-82459c6b0c0a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:43.603509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:43.626254 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103c094-157e-4e3f-9b79-82459c6b0c0a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:43.660644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f245e7-644f-4e0c-ac4b-2d89e260605a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:43.661230 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f245e7-644f-4e0c-ac4b-2d89e260605a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:43.661507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f245e7-644f-4e0c-ac4b-2d89e260605a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:43.661776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f245e7-644f-4e0c-ac4b-2d89e260605a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:43.662519 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f245e7-644f-4e0c-ac4b-2d89e260605a None admin] RBAC: Adding query filter params (role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839, scope.project.id=8324e362dedd4b9b9b8a489eeda4eb43, user.id=92e5ef8da9f7452fa0153c9f5e46bd3a) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:43.662832 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f245e7-644f-4e0c-ac4b-2d89e260605a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:43.664248 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f245e7-644f-4e0c-ac4b-2d89e260605a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:43.669839 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 559/1130] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:43 2022] GET /identity/v3/role_assignments?user.id=92e5ef8da9f7452fa0153c9f5e46bd3a&scope.project.id=8324e362dedd4b9b9b8a489eeda4eb43&role.id=9f8a5b177f4a41fc8bc88ff4f1ce0839 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:45.668336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d72a8f-8613-44c7-8035-324e090fb2c4 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:45.668895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d72a8f-8613-44c7-8035-324e090fb2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:45.668895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d72a8f-8613-44c7-8035-324e090fb2c4 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:45.670212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 572/1131] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:45.675944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc0297-85c9-4dbf-9a5b-65cb32ce4bb6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:45.676366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc0297-85c9-4dbf-9a5b-65cb32ce4bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:45.676488 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc0297-85c9-4dbf-9a5b-65cb32ce4bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:45.694228 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5ccc0297-85c9-4dbf-9a5b-65cb32ce4bb6 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:45.695492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5ccc0297-85c9-4dbf-9a5b-65cb32ce4bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:45.759220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 560/1132] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:45.781199 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d72a8f-8613-44c7-8035-324e090fb2c4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:45.878708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d72a8f-8613-44c7-8035-324e090fb2c4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:45.911961 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dddc7-2fbf-42b0-a32e-8970c0d2e706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:45.912560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dddc7-2fbf-42b0-a32e-8970c0d2e706 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:45.912832 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dddc7-2fbf-42b0-a32e-8970c0d2e706 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:45.913097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dddc7-2fbf-42b0-a32e-8970c0d2e706 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:45.914178 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4dddc7-2fbf-42b0-a32e-8970c0d2e706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:45.915637 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4dddc7-2fbf-42b0-a32e-8970c0d2e706 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:45.917172 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 573/1133] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:45.922933 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc0297-85c9-4dbf-9a5b-65cb32ce4bb6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:45.988877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:46.019229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc0297-85c9-4dbf-9a5b-65cb32ce4bb6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:46.051885 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e9545-9ed8-4dbc-9a59-e2723d9046fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:46.052467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e9545-9ed8-4dbc-9a59-e2723d9046fb None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:46.052747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e9545-9ed8-4dbc-9a59-e2723d9046fb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:46.053014 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e9545-9ed8-4dbc-9a59-e2723d9046fb None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:46.053638 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e9545-9ed8-4dbc-9a59-e2723d9046fb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:46.055069 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e9545-9ed8-4dbc-9a59-e2723d9046fb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:46.084186 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 561/1134] 10.222.0.39 () {60 vars in 1141 bytes} [Mon Aug 8 02:14:45 2022] POST /identity/v3/projects => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:14:48.082761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a2c7c-532e-47eb-90e7-aa728ccbcede None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:48.083311 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a2c7c-532e-47eb-90e7-aa728ccbcede None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:48.083311 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a2c7c-532e-47eb-90e7-aa728ccbcede None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:48.084631 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 574/1135] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:48.089813 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67acb3f1-bebe-41b5-8e23-d99a344ad8ea None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:48.090182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67acb3f1-bebe-41b5-8e23-d99a344ad8ea None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:48.090408 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67acb3f1-bebe-41b5-8e23-d99a344ad8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:48.108006 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-67acb3f1-bebe-41b5-8e23-d99a344ad8ea None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:48.109185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-67acb3f1-bebe-41b5-8e23-d99a344ad8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:48.172735 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 562/1136] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:48.197329 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a2c7c-532e-47eb-90e7-aa728ccbcede None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:48.259035 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:48.296291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a2c7c-532e-47eb-90e7-aa728ccbcede None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:48.329776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def6500-2064-4ea7-8ffd-9359137b64ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:48.330415 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def6500-2064-4ea7-8ffd-9359137b64ab None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:48.330695 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def6500-2064-4ea7-8ffd-9359137b64ab None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:48.330962 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def6500-2064-4ea7-8ffd-9359137b64ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:48.332081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def6500-2064-4ea7-8ffd-9359137b64ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:48.333540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def6500-2064-4ea7-8ffd-9359137b64ab None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:48.335072 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 575/1137] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:48.340754 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67acb3f1-bebe-41b5-8e23-d99a344ad8ea None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:48.390891 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:48.436351 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67acb3f1-bebe-41b5-8e23-d99a344ad8ea None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:48.469158 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b6dec-8118-44d8-8245-66900974c227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:48.469748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b6dec-8118-44d8-8245-66900974c227 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:48.470022 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b6dec-8118-44d8-8245-66900974c227 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:48.470315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b6dec-8118-44d8-8245-66900974c227 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:48.470962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b6dec-8118-44d8-8245-66900974c227 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:48.472415 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b6dec-8118-44d8-8245-66900974c227 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:48.492629 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 563/1138] 10.222.0.39 () {60 vars in 1132 bytes} [Mon Aug 8 02:14:48 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:14:50.517473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba4f4a-3962-43bb-b645-a6fd9b56545b None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:50.518466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba4f4a-3962-43bb-b645-a6fd9b56545b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:50.518567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba4f4a-3962-43bb-b645-a6fd9b56545b None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:50.519893 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 576/1139] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:50.525255 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3779842-5e22-4f15-8149-e8578be20115 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:50.525618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3779842-5e22-4f15-8149-e8578be20115 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:50.525809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3779842-5e22-4f15-8149-e8578be20115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:50.544160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f3779842-5e22-4f15-8149-e8578be20115 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:50.545370 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f3779842-5e22-4f15-8149-e8578be20115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:50.608976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 564/1140] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:50.631667 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba4f4a-3962-43bb-b645-a6fd9b56545b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:50.729075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba4f4a-3962-43bb-b645-a6fd9b56545b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:50.756525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:50.761613 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2e89f-49f7-47bb-b963-4bfe520fa907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:50.762217 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2e89f-49f7-47bb-b963-4bfe520fa907 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:50.762521 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2e89f-49f7-47bb-b963-4bfe520fa907 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:50.762790 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2e89f-49f7-47bb-b963-4bfe520fa907 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:50.773021 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb2e89f-49f7-47bb-b963-4bfe520fa907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:50.777614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-8cb2e89f-49f7-47bb-b963-4bfe520fa907 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:50.777614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:50.777614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:50.777614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:50.777614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:50.777614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:50.777614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:50.777614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:50.777614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:50.777614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:50.777614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:50.777614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:50.777614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:50.777614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:50.777614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:50.777614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:50.778807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:50.778807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:50.778807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:50.778807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:50.778807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:50.778807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:50.778807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:50.778807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:50.778807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:50.778807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:50.778807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:50.778807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:50.778807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:50.778807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:50.778807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:50.778807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:50.779708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:50.779708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:50.779708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:50.779708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:50.779708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:50.779708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:50.779708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:50.779708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:50.779708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:50.779708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:50.779708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:50.779708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:50.779708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 577/1141] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:14:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:50.785194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3779842-5e22-4f15-8149-e8578be20115 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:50.822889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:50.880415 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3779842-5e22-4f15-8149-e8578be20115 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:50.913243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d422b-a595-40e0-8940-654b46c109d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:50.913823 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d422b-a595-40e0-8940-654b46c109d7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:50.914096 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d422b-a595-40e0-8940-654b46c109d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:50.914390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d422b-a595-40e0-8940-654b46c109d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:50.915038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d422b-a595-40e0-8940-654b46c109d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:50.915336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d422b-a595-40e0-8940-654b46c109d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:50.916685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d422b-a595-40e0-8940-654b46c109d7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:50.924310 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 565/1142] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:50.929914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2e89f-49f7-47bb-b963-4bfe520fa907 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:51.025739 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2e89f-49f7-47bb-b963-4bfe520fa907 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:51.057495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79772521-fa99-417b-8eec-4b0d6722e704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:51.058089 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79772521-fa99-417b-8eec-4b0d6722e704 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:51.058398 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79772521-fa99-417b-8eec-4b0d6722e704 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:51.058673 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79772521-fa99-417b-8eec-4b0d6722e704 None admin] PATH_INFO: `/v3/users/6cb2ae79ac0c444c8621cf809af1c372` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:51.072005 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79772521-fa99-417b-8eec-4b0d6722e704 None admin] RBAC: Authorizing `identity:get_user(user_id=6cb2ae79ac0c444c8621cf809af1c372)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:51.073421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79772521-fa99-417b-8eec-4b0d6722e704 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:51.075007 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 578/1143] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:50 2022] GET /identity/v3/users/6cb2ae79ac0c444c8621cf809af1c372 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:51.081239 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d422b-a595-40e0-8940-654b46c109d7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:51.174993 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:51.178075 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d422b-a595-40e0-8940-654b46c109d7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:51.210601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce83777-e40c-4365-8af0-986e63fce20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:51.211179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce83777-e40c-4365-8af0-986e63fce20f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:51.211457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce83777-e40c-4365-8af0-986e63fce20f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:51.211742 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce83777-e40c-4365-8af0-986e63fce20f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:51.275654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce83777-e40c-4365-8af0-986e63fce20f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:51.277159 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce83777-e40c-4365-8af0-986e63fce20f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:51.366582 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:51.370456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 566/1144] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:14:51.376091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79772521-fa99-417b-8eec-4b0d6722e704 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:51.426044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:51.474148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79772521-fa99-417b-8eec-4b0d6722e704 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:51.507123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a66a27-745f-409d-be2a-0a8e82a30c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:51.507734 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a66a27-745f-409d-be2a-0a8e82a30c4b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:51.508012 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a66a27-745f-409d-be2a-0a8e82a30c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:51.508283 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a66a27-745f-409d-be2a-0a8e82a30c4b None admin] PATH_INFO: `/v3/projects/51c7fdf5fd464967b399ef4f512eac69` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:51.509391 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a66a27-745f-409d-be2a-0a8e82a30c4b None admin] RBAC: Authorizing `identity:get_project(project_id=51c7fdf5fd464967b399ef4f512eac69)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:51.510847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a66a27-745f-409d-be2a-0a8e82a30c4b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:51.512453 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 579/1145] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:51 2022] GET /identity/v3/projects/51c7fdf5fd464967b399ef4f512eac69 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:51.518637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce83777-e40c-4365-8af0-986e63fce20f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:51.616903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce83777-e40c-4365-8af0-986e63fce20f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:51.648651 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fa4a3-bf69-41d6-aad6-544d2addf701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:51.649234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fa4a3-bf69-41d6-aad6-544d2addf701 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:51.649508 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fa4a3-bf69-41d6-aad6-544d2addf701 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:51.649774 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fa4a3-bf69-41d6-aad6-544d2addf701 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:51.650516 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fa4a3-bf69-41d6-aad6-544d2addf701 None admin] RBAC: Adding query filter params (role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.project.id=51c7fdf5fd464967b399ef4f512eac69, user.id=6cb2ae79ac0c444c8621cf809af1c372) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:51.650823 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fa4a3-bf69-41d6-aad6-544d2addf701 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:51.652237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fa4a3-bf69-41d6-aad6-544d2addf701 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:51.657343 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 567/1146] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:51 2022] GET /identity/v3/role_assignments?user.id=6cb2ae79ac0c444c8621cf809af1c372&scope.project.id=51c7fdf5fd464967b399ef4f512eac69&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:53.596912 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e0ae1-4f89-4194-976b-46e9c936a38e None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:53.597413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e0ae1-4f89-4194-976b-46e9c936a38e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:53.597532 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e0ae1-4f89-4194-976b-46e9c936a38e None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:53.598838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 580/1147] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:53.604008 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165ae6e-f3de-4bd2-bdd6-c6da5284ed56 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:53.604416 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165ae6e-f3de-4bd2-bdd6-c6da5284ed56 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:53.604556 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165ae6e-f3de-4bd2-bdd6-c6da5284ed56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:53.622673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7165ae6e-f3de-4bd2-bdd6-c6da5284ed56 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:53.623853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7165ae6e-f3de-4bd2-bdd6-c6da5284ed56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:53.680485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:53.689615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 568/1148] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:53.712148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e0ae1-4f89-4194-976b-46e9c936a38e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:53.810056 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e0ae1-4f89-4194-976b-46e9c936a38e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:53.827046 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:53.843228 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc52e7-3846-4bf9-8429-cb28151b82be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:53.843833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc52e7-3846-4bf9-8429-cb28151b82be None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:53.844111 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc52e7-3846-4bf9-8429-cb28151b82be None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:53.844381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc52e7-3846-4bf9-8429-cb28151b82be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:53.854543 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc52e7-3846-4bf9-8429-cb28151b82be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:53.859081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-2fdc52e7-3846-4bf9-8429-cb28151b82be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:53.859081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:53.859081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:53.859081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:53.859081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:53.859081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:53.859081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:53.859081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:53.859081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:53.859081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:53.859081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:53.859081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:53.859081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:53.859081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:53.859081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:53.859081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:53.860152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:53.860152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:53.860152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:53.860152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:53.860152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:53.860152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:53.860152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:53.860152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:53.860152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:53.860152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:53.860152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:53.860152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:53.860152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:53.860152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:53.860152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:53.860152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:53.861198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:53.861198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:53.861198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:53.861198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:53.861198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:53.861198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:53.861198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:53.861198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:53.861198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:53.861198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:53.861198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:53.861198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:14:53.861198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 581/1149] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:14:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:53.866746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165ae6e-f3de-4bd2-bdd6-c6da5284ed56 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:53.961405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165ae6e-f3de-4bd2-bdd6-c6da5284ed56 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:53.994455 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531ac87-9cd0-488f-944b-4b5f0be4dfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:53.994988 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531ac87-9cd0-488f-944b-4b5f0be4dfa5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:53.995266 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531ac87-9cd0-488f-944b-4b5f0be4dfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:53.995533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531ac87-9cd0-488f-944b-4b5f0be4dfa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:53.996184 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531ac87-9cd0-488f-944b-4b5f0be4dfa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:53.996487 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531ac87-9cd0-488f-944b-4b5f0be4dfa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:53.997839 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531ac87-9cd0-488f-944b-4b5f0be4dfa5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:54.005540 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 569/1150] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:54.011210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc52e7-3846-4bf9-8429-cb28151b82be None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:54.107693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc52e7-3846-4bf9-8429-cb28151b82be None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:54.139897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fef5d2-5f1a-411b-88d1-8094f6ab7cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:54.140497 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fef5d2-5f1a-411b-88d1-8094f6ab7cec None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:54.140775 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fef5d2-5f1a-411b-88d1-8094f6ab7cec None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:54.141045 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fef5d2-5f1a-411b-88d1-8094f6ab7cec None admin] PATH_INFO: `/v3/users/6cb2ae79ac0c444c8621cf809af1c372` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:54.142148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fef5d2-5f1a-411b-88d1-8094f6ab7cec None admin] RBAC: Authorizing `identity:get_user(user_id=6cb2ae79ac0c444c8621cf809af1c372)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:54.144101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fef5d2-5f1a-411b-88d1-8094f6ab7cec None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:54.145649 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 582/1151] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:54 2022] GET /identity/v3/users/6cb2ae79ac0c444c8621cf809af1c372 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:54.151139 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531ac87-9cd0-488f-944b-4b5f0be4dfa5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:54.206404 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:54.249169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531ac87-9cd0-488f-944b-4b5f0be4dfa5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:54.281329 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f5d28-3a3d-4ad5-b3f2-c5ea987a8807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:54.281932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f5d28-3a3d-4ad5-b3f2-c5ea987a8807 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:54.282210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f5d28-3a3d-4ad5-b3f2-c5ea987a8807 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:54.282506 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f5d28-3a3d-4ad5-b3f2-c5ea987a8807 None admin] PATH_INFO: `/v3/projects/51c7fdf5fd464967b399ef4f512eac69` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:54.284178 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f5d28-3a3d-4ad5-b3f2-c5ea987a8807 None admin] RBAC: Authorizing `identity:get_project(project_id=51c7fdf5fd464967b399ef4f512eac69)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:54.285594 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f5d28-3a3d-4ad5-b3f2-c5ea987a8807 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:54.287213 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 570/1152] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:54 2022] GET /identity/v3/projects/51c7fdf5fd464967b399ef4f512eac69 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:54.293086 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fef5d2-5f1a-411b-88d1-8094f6ab7cec None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:54.338361 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:54.390226 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fef5d2-5f1a-411b-88d1-8094f6ab7cec None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:54.423157 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d509d32-b01f-431d-9044-bbbebdc90278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:54.423799 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d509d32-b01f-431d-9044-bbbebdc90278 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:54.424113 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d509d32-b01f-431d-9044-bbbebdc90278 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:54.424386 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d509d32-b01f-431d-9044-bbbebdc90278 None admin] PATH_INFO: `/v3/projects/51c7fdf5fd464967b399ef4f512eac69/users/6cb2ae79ac0c444c8621cf809af1c372/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:54.426291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d509d32-b01f-431d-9044-bbbebdc90278 None admin] RBAC: Authorizing `identity:create_grant(project_id=51c7fdf5fd464967b399ef4f512eac69, user_id=6cb2ae79ac0c444c8621cf809af1c372, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:54.427709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d509d32-b01f-431d-9044-bbbebdc90278 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:54.440552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 583/1153] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:14:54 2022] PUT /identity/v3/projects/51c7fdf5fd464967b399ef4f512eac69/users/6cb2ae79ac0c444c8621cf809af1c372/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:14:56.408049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f7238-37f9-42f0-b5d7-1e0c3b9c28fb None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:56.408573 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f7238-37f9-42f0-b5d7-1e0c3b9c28fb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:56.408686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f7238-37f9-42f0-b5d7-1e0c3b9c28fb None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:56.409974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 571/1154] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:56.415176 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8cb42-9792-4e5a-8e59-a98c108e5d97 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:56.415555 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8cb42-9792-4e5a-8e59-a98c108e5d97 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:56.415729 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8cb42-9792-4e5a-8e59-a98c108e5d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:56.433840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-3fe8cb42-9792-4e5a-8e59-a98c108e5d97 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:56.435320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3fe8cb42-9792-4e5a-8e59-a98c108e5d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:56.499413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 584/1155] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:56.523238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f7238-37f9-42f0-b5d7-1e0c3b9c28fb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:56.626047 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f7238-37f9-42f0-b5d7-1e0c3b9c28fb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:56.648402 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:56.659185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c7d54-3c32-47e1-bde1-25deab72ba2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:56.659732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c7d54-3c32-47e1-bde1-25deab72ba2f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:56.660005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c7d54-3c32-47e1-bde1-25deab72ba2f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:56.660270 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c7d54-3c32-47e1-bde1-25deab72ba2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:56.670224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89c7d54-3c32-47e1-bde1-25deab72ba2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:56.675312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b89c7d54-3c32-47e1-bde1-25deab72ba2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:56.675312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:56.675312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:56.675312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:56.675312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:56.675312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:56.675312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:56.675312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:56.675312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:56.675312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:56.675312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:56.675312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:56.675312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:56.675312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:56.675312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:56.675312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:56.676407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:56.676407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:56.676407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:56.676407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:56.676407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:56.676407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:56.676407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:56.676407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:56.676407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:56.676407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:56.676407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:56.676407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:56.676407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:56.676407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:56.676407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:56.676407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:56.677453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:56.677453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:56.677453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:56.677453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:56.677453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:56.677453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:56.677453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:56.677453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:56.677453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:56.677453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:56.677453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:56.677453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:14:56.677453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 572/1156] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:14:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:56.682633 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8cb42-9792-4e5a-8e59-a98c108e5d97 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:56.716400 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:56.780582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8cb42-9792-4e5a-8e59-a98c108e5d97 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:56.813955 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7316a22-55e4-4ef0-a113-18d42d22caff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:56.814593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7316a22-55e4-4ef0-a113-18d42d22caff None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:56.814875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7316a22-55e4-4ef0-a113-18d42d22caff None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:56.815144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7316a22-55e4-4ef0-a113-18d42d22caff None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:56.815796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7316a22-55e4-4ef0-a113-18d42d22caff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:56.816099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7316a22-55e4-4ef0-a113-18d42d22caff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:56.817444 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7316a22-55e4-4ef0-a113-18d42d22caff None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:56.825228 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 585/1157] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:56.830820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c7d54-3c32-47e1-bde1-25deab72ba2f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:56.926408 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c7d54-3c32-47e1-bde1-25deab72ba2f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:56.957793 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbadc8-8577-42ee-b394-34b18ecc3e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:56.958419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbadc8-8577-42ee-b394-34b18ecc3e88 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:56.958696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbadc8-8577-42ee-b394-34b18ecc3e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:56.958963 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbadc8-8577-42ee-b394-34b18ecc3e88 None admin] PATH_INFO: `/v3/users/6cb2ae79ac0c444c8621cf809af1c372` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:56.960051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbadc8-8577-42ee-b394-34b18ecc3e88 None admin] RBAC: Authorizing `identity:get_user(user_id=6cb2ae79ac0c444c8621cf809af1c372)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:56.961461 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbadc8-8577-42ee-b394-34b18ecc3e88 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:56.963030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 573/1158] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:56 2022] GET /identity/v3/users/6cb2ae79ac0c444c8621cf809af1c372 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:56.968490 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7316a22-55e4-4ef0-a113-18d42d22caff None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:57.062750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:57.067161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7316a22-55e4-4ef0-a113-18d42d22caff None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:57.101457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a71037-69c1-4eb7-9ad6-1887592ce19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:57.102047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a71037-69c1-4eb7-9ad6-1887592ce19f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:57.102357 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a71037-69c1-4eb7-9ad6-1887592ce19f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:57.102632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a71037-69c1-4eb7-9ad6-1887592ce19f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:57.168109 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a71037-69c1-4eb7-9ad6-1887592ce19f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:57.169546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a71037-69c1-4eb7-9ad6-1887592ce19f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:57.259930 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:57.264590 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 586/1159] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:14:57.270721 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbadc8-8577-42ee-b394-34b18ecc3e88 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:57.321710 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:57.367553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbadc8-8577-42ee-b394-34b18ecc3e88 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:57.399279 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe4bc5-e46c-442b-b22d-35e1285894fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:57.399880 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe4bc5-e46c-442b-b22d-35e1285894fa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:57.400154 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe4bc5-e46c-442b-b22d-35e1285894fa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:57.400417 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe4bc5-e46c-442b-b22d-35e1285894fa None admin] PATH_INFO: `/v3/projects/51c7fdf5fd464967b399ef4f512eac69` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:57.401502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe4bc5-e46c-442b-b22d-35e1285894fa None admin] RBAC: Authorizing `identity:get_project(project_id=51c7fdf5fd464967b399ef4f512eac69)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:57.402937 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe4bc5-e46c-442b-b22d-35e1285894fa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:57.404528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 574/1160] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:57 2022] GET /identity/v3/projects/51c7fdf5fd464967b399ef4f512eac69 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:57.411177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a71037-69c1-4eb7-9ad6-1887592ce19f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:57.506875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a71037-69c1-4eb7-9ad6-1887592ce19f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:57.540152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444048a-9d29-4fac-bb78-3f3209ad07b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:57.540680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444048a-9d29-4fac-bb78-3f3209ad07b0 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:57.540957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444048a-9d29-4fac-bb78-3f3209ad07b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:57.541228 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444048a-9d29-4fac-bb78-3f3209ad07b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:57.541943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444048a-9d29-4fac-bb78-3f3209ad07b0 None admin] RBAC: Adding query filter params (role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.project.id=51c7fdf5fd464967b399ef4f512eac69, user.id=6cb2ae79ac0c444c8621cf809af1c372) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:57.542274 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444048a-9d29-4fac-bb78-3f3209ad07b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:57.543695 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444048a-9d29-4fac-bb78-3f3209ad07b0 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:57.549261 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 587/1161] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:57 2022] GET /identity/v3/role_assignments?user.id=6cb2ae79ac0c444c8621cf809af1c372&scope.project.id=51c7fdf5fd464967b399ef4f512eac69&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:59.540859 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b8105-7cdc-4656-a1cc-64c381fedc5d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:59.541404 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b8105-7cdc-4656-a1cc-64c381fedc5d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:59.541404 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b8105-7cdc-4656-a1cc-64c381fedc5d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:59.542757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 575/1162] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:59.547937 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd74a95-6562-4e29-9710-628a2c7b0300 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:59.548365 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd74a95-6562-4e29-9710-628a2c7b0300 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:59.548493 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd74a95-6562-4e29-9710-628a2c7b0300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:59.566421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-dbd74a95-6562-4e29-9710-628a2c7b0300 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:59.567616 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dbd74a95-6562-4e29-9710-628a2c7b0300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:59.622346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:59.633010 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 588/1163] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:14:59.655409 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b8105-7cdc-4656-a1cc-64c381fedc5d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:59.753122 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b8105-7cdc-4656-a1cc-64c381fedc5d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:59.772387 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:59.786560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b87ad3-4a47-4737-bcd2-b1d035bdbfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:59.787154 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b87ad3-4a47-4737-bcd2-b1d035bdbfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:59.787434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b87ad3-4a47-4737-bcd2-b1d035bdbfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:59.787704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b87ad3-4a47-4737-bcd2-b1d035bdbfb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:59.852891 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b87ad3-4a47-4737-bcd2-b1d035bdbfb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:59.854358 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b87ad3-4a47-4737-bcd2-b1d035bdbfb1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:59.949779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 576/1164] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:14:59.956378 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd74a95-6562-4e29-9710-628a2c7b0300 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:00.050439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd74a95-6562-4e29-9710-628a2c7b0300 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:00.083496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe2c28-d87c-4520-ad23-a1c3dac747c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:00.084104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe2c28-d87c-4520-ad23-a1c3dac747c8 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:00.084381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe2c28-d87c-4520-ad23-a1c3dac747c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:00.084651 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe2c28-d87c-4520-ad23-a1c3dac747c8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:00.091063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe2c28-d87c-4520-ad23-a1c3dac747c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:00.092489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe2c28-d87c-4520-ad23-a1c3dac747c8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:00.103392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-cebe2c28-d87c-4520-ad23-a1c3dac747c8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 02:15:00.103392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:00.103392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 08 02:15:00.103392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 08 02:15:00.103392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:15:00.103392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:15:00.103392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:15:00.103392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:15:00.103392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 08 02:15:00.103392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:15:00.103392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:00.103392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:15:00.103392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:00.103392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:00.103392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:00.103392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:00.103392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:00.104582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:00.104582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:00.104582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:00.104582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:00.104582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:00.104582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:00.104582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:00.104582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:00.104582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:00.104582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:00.104582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:00.104582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:00.104582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:00.104582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:00.104582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:00.104582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:00.105459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:00.105459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:00.105459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:00.105459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:00.105459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:00.105459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:00.105459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:00.105459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:00.105459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:00.105459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:00.105459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 08 02:15:00.105459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:00.105459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 02:15:00.105459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:15:00.105459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 589/1165] 10.222.0.39 () {56 vars in 1117 bytes} [Mon Aug 8 02:14:59 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:00.112955 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b87ad3-4a47-4737-bcd2-b1d035bdbfb1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:00.134284 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:00.209739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b87ad3-4a47-4737-bcd2-b1d035bdbfb1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:00.242256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a8d12-62ec-45f0-b289-bc54d5e72917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:00.242864 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a8d12-62ec-45f0-b289-bc54d5e72917 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:00.243139 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a8d12-62ec-45f0-b289-bc54d5e72917 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:00.243406 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a8d12-62ec-45f0-b289-bc54d5e72917 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:00.244031 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a8d12-62ec-45f0-b289-bc54d5e72917 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:00.244329 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a8d12-62ec-45f0-b289-bc54d5e72917 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:00.245678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a8d12-62ec-45f0-b289-bc54d5e72917 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:00.253066 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 577/1166] 10.222.0.39 () {56 vars in 1115 bytes} [Mon Aug 8 02:15:00 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:02.210589 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f77240-49e9-4e28-9b35-cc9e40f556b7 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:02.211134 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f77240-49e9-4e28-9b35-cc9e40f556b7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:02.211134 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f77240-49e9-4e28-9b35-cc9e40f556b7 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:02.212463 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 590/1167] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:02.217740 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3923e66-2884-48cf-8e4f-ee8fab115587 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:02.218150 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3923e66-2884-48cf-8e4f-ee8fab115587 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:02.218296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3923e66-2884-48cf-8e4f-ee8fab115587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:02.236354 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d3923e66-2884-48cf-8e4f-ee8fab115587 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:02.237513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d3923e66-2884-48cf-8e4f-ee8fab115587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:02.301985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 578/1168] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:15:02.324120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f77240-49e9-4e28-9b35-cc9e40f556b7 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:02.376057 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:02.422583 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f77240-49e9-4e28-9b35-cc9e40f556b7 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:02.455074 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d498471-fca0-448c-b1e5-733a8a35f728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:02.455660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d498471-fca0-448c-b1e5-733a8a35f728 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:02.455938 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d498471-fca0-448c-b1e5-733a8a35f728 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:02.456210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d498471-fca0-448c-b1e5-733a8a35f728 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:02.456861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d498471-fca0-448c-b1e5-733a8a35f728 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:02.458764 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d498471-fca0-448c-b1e5-733a8a35f728 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:02.474968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 591/1169] 10.222.0.39 () {60 vars in 1138 bytes} [Mon Aug 8 02:15:02 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:15:04.483554 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f2c0c-c86a-4ea2-b723-62953fff8632 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:04.484041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f2c0c-c86a-4ea2-b723-62953fff8632 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:04.484146 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f2c0c-c86a-4ea2-b723-62953fff8632 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:04.485473 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 579/1170] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:04.491200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27445d78-dea6-418d-b0a2-79b750872d86 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:04.491619 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27445d78-dea6-418d-b0a2-79b750872d86 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:04.491740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27445d78-dea6-418d-b0a2-79b750872d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:04.509128 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-27445d78-dea6-418d-b0a2-79b750872d86 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:04.510324 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-27445d78-dea6-418d-b0a2-79b750872d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:04.581118 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 592/1171] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:04.604752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f2c0c-c86a-4ea2-b723-62953fff8632 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:04.620636 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:04.704037 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f2c0c-c86a-4ea2-b723-62953fff8632 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:04.736958 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814d7bf-c2e4-433d-a266-341729efa78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:04.737563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814d7bf-c2e4-433d-a266-341729efa78e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:04.737840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814d7bf-c2e4-433d-a266-341729efa78e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:04.738130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814d7bf-c2e4-433d-a266-341729efa78e None admin] PATH_INFO: `/v3/domains/e578a9ba51d54d1485c1675203c85648` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:04.748121 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814d7bf-c2e4-433d-a266-341729efa78e None admin] RBAC: Authorizing `identity:get_domain(domain_id=e578a9ba51d54d1485c1675203c85648)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:04.749559 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814d7bf-c2e4-433d-a266-341729efa78e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:04.751158 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 580/1172] 10.222.0.39 () {56 vars in 1183 bytes} [Mon Aug 8 02:15:04 2022] GET /identity/v3/domains/e578a9ba51d54d1485c1675203c85648 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:04.757031 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27445d78-dea6-418d-b0a2-79b750872d86 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:04.798622 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:04.853881 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27445d78-dea6-418d-b0a2-79b750872d86 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:04.887087 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56da92-5d98-4c77-95ca-2654cf650573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:04.887672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56da92-5d98-4c77-95ca-2654cf650573 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:04.887950 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56da92-5d98-4c77-95ca-2654cf650573 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:04.888217 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56da92-5d98-4c77-95ca-2654cf650573 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:04.888845 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56da92-5d98-4c77-95ca-2654cf650573 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:04.890276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56da92-5d98-4c77-95ca-2654cf650573 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:04.915727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 593/1173] 10.222.0.39 () {60 vars in 1141 bytes} [Mon Aug 8 02:15:04 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:15:06.944135 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdc81f-9f5a-4b2f-acc3-78364335d62a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:06.944685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdc81f-9f5a-4b2f-acc3-78364335d62a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:06.944685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdc81f-9f5a-4b2f-acc3-78364335d62a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:06.946038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 581/1174] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:06.951119 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f234e1-19dc-4708-bfe2-a23b5f99e537 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:06.951542 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f234e1-19dc-4708-bfe2-a23b5f99e537 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:06.951660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f234e1-19dc-4708-bfe2-a23b5f99e537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:06.969519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-48f234e1-19dc-4708-bfe2-a23b5f99e537 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:06.970723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-48f234e1-19dc-4708-bfe2-a23b5f99e537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:07.033247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 594/1175] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:07.057490 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdc81f-9f5a-4b2f-acc3-78364335d62a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:07.131301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:07.157446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdc81f-9f5a-4b2f-acc3-78364335d62a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:07.189972 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6ff74-6d9d-4acc-9009-a754939ce5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:07.190606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6ff74-6d9d-4acc-9009-a754939ce5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:07.190887 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6ff74-6d9d-4acc-9009-a754939ce5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:07.191158 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6ff74-6d9d-4acc-9009-a754939ce5c7 None admin] PATH_INFO: `/v3/domains/e578a9ba51d54d1485c1675203c85648` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:07.192269 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6ff74-6d9d-4acc-9009-a754939ce5c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e578a9ba51d54d1485c1675203c85648)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:07.193698 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6ff74-6d9d-4acc-9009-a754939ce5c7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:07.195332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 582/1176] 10.222.0.39 () {56 vars in 1183 bytes} [Mon Aug 8 02:15:07 2022] GET /identity/v3/domains/e578a9ba51d54d1485c1675203c85648 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:07.200872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f234e1-19dc-4708-bfe2-a23b5f99e537 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:07.225553 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:07.298406 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f234e1-19dc-4708-bfe2-a23b5f99e537 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:07.331560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab7657-d422-4856-bd8a-0bf34c9a49cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:07.332761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab7657-d422-4856-bd8a-0bf34c9a49cb None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:07.333037 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab7657-d422-4856-bd8a-0bf34c9a49cb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:07.333305 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab7657-d422-4856-bd8a-0bf34c9a49cb None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:07.333940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab7657-d422-4856-bd8a-0bf34c9a49cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:07.335384 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab7657-d422-4856-bd8a-0bf34c9a49cb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:07.355147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 595/1177] 10.222.0.39 () {60 vars in 1132 bytes} [Mon Aug 8 02:15:07 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:15:09.366512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47012264-b12d-418a-b071-1a05660cf5ff None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:09.367081 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47012264-b12d-418a-b071-1a05660cf5ff None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:09.367081 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47012264-b12d-418a-b071-1a05660cf5ff None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:09.368364 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 583/1178] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:09.373386 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b78c48-3908-4a46-9e90-9767fd04eefa None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:09.373768 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b78c48-3908-4a46-9e90-9767fd04eefa None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:09.373936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b78c48-3908-4a46-9e90-9767fd04eefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:09.391445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-43b78c48-3908-4a46-9e90-9767fd04eefa None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:09.392617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-43b78c48-3908-4a46-9e90-9767fd04eefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:09.458590 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 596/1179] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:09.481770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47012264-b12d-418a-b071-1a05660cf5ff None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:09.579512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47012264-b12d-418a-b071-1a05660cf5ff None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:09.611208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397722f8-0162-42f3-ba44-bd345b5c51c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:09.611803 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397722f8-0162-42f3-ba44-bd345b5c51c1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:09.612076 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397722f8-0162-42f3-ba44-bd345b5c51c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:09.612339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397722f8-0162-42f3-ba44-bd345b5c51c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:09.623011 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397722f8-0162-42f3-ba44-bd345b5c51c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:09.627586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-397722f8-0162-42f3-ba44-bd345b5c51c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:09.627586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:09.627586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:09.627586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:09.627586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:09.627586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:09.627586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:09.627586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:09.627586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:09.627586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:09.627586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:09.627586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:09.627586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:09.627586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:09.627586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:09.627586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:09.629137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:09.629137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:09.629137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:09.629137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:09.629137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:09.629137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:09.629137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:09.629137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:09.629137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:09.629137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:09.629137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:09.629137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:09.629137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:09.629137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:09.629137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:09.629137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:09.630349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:09.630349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:09.630349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:09.630349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:09.630349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:09.630349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:09.630349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:09.630349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:09.630349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:09.630349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:09.630349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:09.630349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:15:09.630349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 584/1180] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:15:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:09.635004 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b78c48-3908-4a46-9e90-9767fd04eefa None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:09.646994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:09.732688 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b78c48-3908-4a46-9e90-9767fd04eefa None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:09.765508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f782867-1dd3-4a2e-ab3a-626e222760a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:09.766096 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f782867-1dd3-4a2e-ab3a-626e222760a5 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:09.766405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f782867-1dd3-4a2e-ab3a-626e222760a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:09.766679 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f782867-1dd3-4a2e-ab3a-626e222760a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:09.767330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f782867-1dd3-4a2e-ab3a-626e222760a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:09.767631 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f782867-1dd3-4a2e-ab3a-626e222760a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:09.768985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f782867-1dd3-4a2e-ab3a-626e222760a5 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:09.777361 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 597/1181] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:15:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:09.782953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397722f8-0162-42f3-ba44-bd345b5c51c1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:09.785835 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:09.880265 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397722f8-0162-42f3-ba44-bd345b5c51c1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:09.916341 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b176b3-f612-41fe-93ff-c46205a08b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:09.916954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b176b3-f612-41fe-93ff-c46205a08b61 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:09.917231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b176b3-f612-41fe-93ff-c46205a08b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:09.917500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b176b3-f612-41fe-93ff-c46205a08b61 None admin] PATH_INFO: `/v3/users/50ecfbece5ba4fbf811003296fb76d4b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:09.930624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b176b3-f612-41fe-93ff-c46205a08b61 None admin] RBAC: Authorizing `identity:get_user(user_id=50ecfbece5ba4fbf811003296fb76d4b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:09.932098 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b176b3-f612-41fe-93ff-c46205a08b61 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:09.933676 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 585/1182] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:15:09 2022] GET /identity/v3/users/50ecfbece5ba4fbf811003296fb76d4b => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:09.939665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f782867-1dd3-4a2e-ab3a-626e222760a5 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:10.007758 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:10.037131 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f782867-1dd3-4a2e-ab3a-626e222760a5 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:10.069736 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2ba9d-45b3-4b8e-a109-cae1ab3ab51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:10.070344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2ba9d-45b3-4b8e-a109-cae1ab3ab51a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:10.070621 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2ba9d-45b3-4b8e-a109-cae1ab3ab51a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:10.070888 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2ba9d-45b3-4b8e-a109-cae1ab3ab51a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:10.135509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf2ba9d-45b3-4b8e-a109-cae1ab3ab51a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:10.136871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf2ba9d-45b3-4b8e-a109-cae1ab3ab51a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:10.201617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:10.231322 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 598/1183] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:15:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:15:10.237169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b176b3-f612-41fe-93ff-c46205a08b61 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:10.295538 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:10.335604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b176b3-f612-41fe-93ff-c46205a08b61 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:10.367661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c0801-3c07-4546-b7a0-e7f3fe63e1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:10.368267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c0801-3c07-4546-b7a0-e7f3fe63e1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:10.368543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c0801-3c07-4546-b7a0-e7f3fe63e1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:10.368811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c0801-3c07-4546-b7a0-e7f3fe63e1f1 None admin] PATH_INFO: `/v3/projects/984ff88e6a0147bebaf29974f9787961` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:10.369903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c0801-3c07-4546-b7a0-e7f3fe63e1f1 None admin] RBAC: Authorizing `identity:get_project(project_id=984ff88e6a0147bebaf29974f9787961)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:10.371339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c0801-3c07-4546-b7a0-e7f3fe63e1f1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:10.372933 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 586/1184] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:15:10 2022] GET /identity/v3/projects/984ff88e6a0147bebaf29974f9787961 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:10.379352 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2ba9d-45b3-4b8e-a109-cae1ab3ab51a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:10.475589 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2ba9d-45b3-4b8e-a109-cae1ab3ab51a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:10.507988 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25117a-de93-4d7b-a19e-39cee8f4ee6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:10.508576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25117a-de93-4d7b-a19e-39cee8f4ee6a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:10.508855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25117a-de93-4d7b-a19e-39cee8f4ee6a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:10.509128 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25117a-de93-4d7b-a19e-39cee8f4ee6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:10.509842 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c25117a-de93-4d7b-a19e-39cee8f4ee6a None admin] RBAC: Adding query filter params (role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.project.id=984ff88e6a0147bebaf29974f9787961, user.id=50ecfbece5ba4fbf811003296fb76d4b) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:10.510156 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c25117a-de93-4d7b-a19e-39cee8f4ee6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:10.511597 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c25117a-de93-4d7b-a19e-39cee8f4ee6a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:10.516781 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 599/1185] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:15:10 2022] GET /identity/v3/role_assignments?user.id=50ecfbece5ba4fbf811003296fb76d4b&scope.project.id=984ff88e6a0147bebaf29974f9787961&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:12.476935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed794f91-b7dc-40e2-a028-120ac8d21a75 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:12.477394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed794f91-b7dc-40e2-a028-120ac8d21a75 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:12.477487 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed794f91-b7dc-40e2-a028-120ac8d21a75 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:12.478840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 587/1186] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:12.483989 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8488cbb-cec6-497c-a997-7c8713cd32cb None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:12.484369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8488cbb-cec6-497c-a997-7c8713cd32cb None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:12.484530 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8488cbb-cec6-497c-a997-7c8713cd32cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:12.502580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-e8488cbb-cec6-497c-a997-7c8713cd32cb None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:12.503760 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e8488cbb-cec6-497c-a997-7c8713cd32cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:12.530736 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:12.568358 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 600/1187] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:12.589702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed794f91-b7dc-40e2-a028-120ac8d21a75 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:12.687946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed794f91-b7dc-40e2-a028-120ac8d21a75 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:12.712847 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:12.720741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e87206-081b-40fd-9404-ed3b67de2ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:12.721342 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e87206-081b-40fd-9404-ed3b67de2ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:12.721618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e87206-081b-40fd-9404-ed3b67de2ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:12.721887 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e87206-081b-40fd-9404-ed3b67de2ae5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:12.732072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e87206-081b-40fd-9404-ed3b67de2ae5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:12.736685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-42e87206-081b-40fd-9404-ed3b67de2ae5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:12.736685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:12.736685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:12.736685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:12.736685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:12.736685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:12.736685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:12.736685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:12.736685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:12.736685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:12.736685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:12.736685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:12.736685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:12.736685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:12.736685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:12.736685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:12.737734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:12.737734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:12.737734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:12.737734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:12.737734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:12.737734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:12.737734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:12.737734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:12.737734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:12.737734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:12.737734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:12.737734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:12.737734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:12.737734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:12.737734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:12.737734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:12.739563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:12.739563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:12.739563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:12.739563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:12.739563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:12.739563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:12.739563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:12.739563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:12.739563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:12.739563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:12.739563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:12.739563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:15:12.739563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 588/1188] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:15:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:12.744136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8488cbb-cec6-497c-a997-7c8713cd32cb None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:12.838091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8488cbb-cec6-497c-a997-7c8713cd32cb None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:12.871986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5bbf9-25fc-410b-8925-366c77b8b5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:12.872572 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5bbf9-25fc-410b-8925-366c77b8b5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:12.872849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5bbf9-25fc-410b-8925-366c77b8b5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:12.873118 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5bbf9-25fc-410b-8925-366c77b8b5a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:12.873768 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a5bbf9-25fc-410b-8925-366c77b8b5a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:12.874071 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a5bbf9-25fc-410b-8925-366c77b8b5a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:12.875453 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a5bbf9-25fc-410b-8925-366c77b8b5a1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:12.883219 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 601/1189] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:15:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:12.888860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e87206-081b-40fd-9404-ed3b67de2ae5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:12.985889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e87206-081b-40fd-9404-ed3b67de2ae5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:13.017850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6859d-bb2f-4719-aa59-f1bdca94fb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:13.018471 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6859d-bb2f-4719-aa59-f1bdca94fb42 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:13.018748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6859d-bb2f-4719-aa59-f1bdca94fb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:13.019014 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6859d-bb2f-4719-aa59-f1bdca94fb42 None admin] PATH_INFO: `/v3/users/50ecfbece5ba4fbf811003296fb76d4b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:13.020602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6859d-bb2f-4719-aa59-f1bdca94fb42 None admin] RBAC: Authorizing `identity:get_user(user_id=50ecfbece5ba4fbf811003296fb76d4b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:13.022003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6859d-bb2f-4719-aa59-f1bdca94fb42 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:13.023553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 589/1190] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:15:12 2022] GET /identity/v3/users/50ecfbece5ba4fbf811003296fb76d4b => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:13.029023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5bbf9-25fc-410b-8925-366c77b8b5a1 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:13.057240 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:13.127641 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5bbf9-25fc-410b-8925-366c77b8b5a1 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:13.159361 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166e000-56bc-41b5-a7bf-1adb4ac7a1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:13.160551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166e000-56bc-41b5-a7bf-1adb4ac7a1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:13.160823 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166e000-56bc-41b5-a7bf-1adb4ac7a1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:13.161084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166e000-56bc-41b5-a7bf-1adb4ac7a1b6 None admin] PATH_INFO: `/v3/projects/984ff88e6a0147bebaf29974f9787961` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:13.162235 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166e000-56bc-41b5-a7bf-1adb4ac7a1b6 None admin] RBAC: Authorizing `identity:get_project(project_id=984ff88e6a0147bebaf29974f9787961)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:13.163662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166e000-56bc-41b5-a7bf-1adb4ac7a1b6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:13.165245 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 602/1191] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:15:13 2022] GET /identity/v3/projects/984ff88e6a0147bebaf29974f9787961 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:13.171013 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6859d-bb2f-4719-aa59-f1bdca94fb42 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:13.224536 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:13.265725 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6859d-bb2f-4719-aa59-f1bdca94fb42 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:13.298654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec377bfe-0edd-40d6-b818-94d9e22a2208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:13.299273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec377bfe-0edd-40d6-b818-94d9e22a2208 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:13.299546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec377bfe-0edd-40d6-b818-94d9e22a2208 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:13.299814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec377bfe-0edd-40d6-b818-94d9e22a2208 None admin] PATH_INFO: `/v3/projects/984ff88e6a0147bebaf29974f9787961/users/50ecfbece5ba4fbf811003296fb76d4b/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:13.301690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec377bfe-0edd-40d6-b818-94d9e22a2208 None admin] RBAC: Authorizing `identity:create_grant(project_id=984ff88e6a0147bebaf29974f9787961, user_id=50ecfbece5ba4fbf811003296fb76d4b, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:13.303131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec377bfe-0edd-40d6-b818-94d9e22a2208 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:13.315372 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 590/1192] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:15:13 2022] PUT /identity/v3/projects/984ff88e6a0147bebaf29974f9787961/users/50ecfbece5ba4fbf811003296fb76d4b/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:15:15.275731 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494fbd4-7e70-4768-8990-fc34e7513ce7 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:15.276267 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494fbd4-7e70-4768-8990-fc34e7513ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:15.276267 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494fbd4-7e70-4768-8990-fc34e7513ce7 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:15.277599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 603/1193] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:15.282658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9584f8-6bd5-492a-8fec-8d8f5e98d20f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:15.283070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9584f8-6bd5-492a-8fec-8d8f5e98d20f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:15.283204 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9584f8-6bd5-492a-8fec-8d8f5e98d20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:15.301164 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-dc9584f8-6bd5-492a-8fec-8d8f5e98d20f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:15.302374 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dc9584f8-6bd5-492a-8fec-8d8f5e98d20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:15.366450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 591/1194] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:15.389873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494fbd4-7e70-4768-8990-fc34e7513ce7 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:15.484497 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:15.492084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494fbd4-7e70-4768-8990-fc34e7513ce7 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:15.523776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ffdab-31bf-4910-8fb5-62ea08d6ac02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:15.524383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ffdab-31bf-4910-8fb5-62ea08d6ac02 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:15.524658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ffdab-31bf-4910-8fb5-62ea08d6ac02 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:15.524925 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ffdab-31bf-4910-8fb5-62ea08d6ac02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:15.534547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ffdab-31bf-4910-8fb5-62ea08d6ac02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:15.539788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-754ffdab-31bf-4910-8fb5-62ea08d6ac02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:15.539788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:15.539788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:15.539788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:15.539788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:15.539788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:15.539788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:15.539788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:15.539788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:15.539788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:15.539788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:15.539788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:15.539788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:15.539788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:15.539788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:15.539788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:15.540848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:15.540848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:15.540848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:15.540848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:15.540848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:15.540848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:15.540848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:15.540848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:15.540848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:15.540848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:15.540848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:15.540848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:15.540848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:15.540848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:15.540848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:15.540848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:15.542014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:15.542014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:15.542014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:15.542014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:15.542014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:15.542014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:15.542014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:15.542014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:15.542014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:15.542014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:15.542014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:15.542014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:15:15.542014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 604/1195] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:15:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:15.547010 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9584f8-6bd5-492a-8fec-8d8f5e98d20f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:15.591629 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:15.643353 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9584f8-6bd5-492a-8fec-8d8f5e98d20f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:15.676270 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd741b7-438c-4fc1-ac71-43c0991b4214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:15.676851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd741b7-438c-4fc1-ac71-43c0991b4214 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:15.677127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd741b7-438c-4fc1-ac71-43c0991b4214 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:15.677390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd741b7-438c-4fc1-ac71-43c0991b4214 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:15.678028 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd741b7-438c-4fc1-ac71-43c0991b4214 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:15.678356 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd741b7-438c-4fc1-ac71-43c0991b4214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:15.679711 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd741b7-438c-4fc1-ac71-43c0991b4214 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:15.687649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 592/1196] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:15:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:15.693208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ffdab-31bf-4910-8fb5-62ea08d6ac02 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:15.787829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ffdab-31bf-4910-8fb5-62ea08d6ac02 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:15.820148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db8af4-17ef-4254-8340-d423422865bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:15.820737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db8af4-17ef-4254-8340-d423422865bd None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:15.821011 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db8af4-17ef-4254-8340-d423422865bd None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:15.821279 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db8af4-17ef-4254-8340-d423422865bd None admin] PATH_INFO: `/v3/users/50ecfbece5ba4fbf811003296fb76d4b` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:15.822396 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db8af4-17ef-4254-8340-d423422865bd None admin] RBAC: Authorizing `identity:get_user(user_id=50ecfbece5ba4fbf811003296fb76d4b)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:15.823793 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db8af4-17ef-4254-8340-d423422865bd None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:15.825312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 605/1197] 10.222.0.39 () {56 vars in 1177 bytes} [Mon Aug 8 02:15:15 2022] GET /identity/v3/users/50ecfbece5ba4fbf811003296fb76d4b => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:15.830969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd741b7-438c-4fc1-ac71-43c0991b4214 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:15.926207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd741b7-438c-4fc1-ac71-43c0991b4214 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:15.936210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:15.961754 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa389f8-e7f4-49b2-912b-1b7cbdcf2347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:15.962381 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa389f8-e7f4-49b2-912b-1b7cbdcf2347 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:15.962659 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa389f8-e7f4-49b2-912b-1b7cbdcf2347 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:15.962928 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa389f8-e7f4-49b2-912b-1b7cbdcf2347 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:16.026708 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa389f8-e7f4-49b2-912b-1b7cbdcf2347 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:16.028143 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa389f8-e7f4-49b2-912b-1b7cbdcf2347 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:16.121915 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 593/1198] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:15:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:15:16.127812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db8af4-17ef-4254-8340-d423422865bd None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:16.152727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:16.225076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db8af4-17ef-4254-8340-d423422865bd None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:16.257636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436f05f-df29-482a-b7d1-b2517e619acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:16.258240 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436f05f-df29-482a-b7d1-b2517e619acf None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:16.258546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436f05f-df29-482a-b7d1-b2517e619acf None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:16.258816 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436f05f-df29-482a-b7d1-b2517e619acf None admin] PATH_INFO: `/v3/projects/984ff88e6a0147bebaf29974f9787961` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:16.259910 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436f05f-df29-482a-b7d1-b2517e619acf None admin] RBAC: Authorizing `identity:get_project(project_id=984ff88e6a0147bebaf29974f9787961)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:16.261323 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436f05f-df29-482a-b7d1-b2517e619acf None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:16.262962 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 606/1199] 10.222.0.39 () {56 vars in 1186 bytes} [Mon Aug 8 02:15:16 2022] GET /identity/v3/projects/984ff88e6a0147bebaf29974f9787961 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:16.269621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa389f8-e7f4-49b2-912b-1b7cbdcf2347 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:16.279452 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:16.365469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa389f8-e7f4-49b2-912b-1b7cbdcf2347 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:16.398456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f9778-d241-434d-8fc7-8d55bd6b921f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:16.398985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f9778-d241-434d-8fc7-8d55bd6b921f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:16.399264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f9778-d241-434d-8fc7-8d55bd6b921f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:16.399534 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f9778-d241-434d-8fc7-8d55bd6b921f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:16.400241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f9778-d241-434d-8fc7-8d55bd6b921f None admin] RBAC: Adding query filter params (role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.project.id=984ff88e6a0147bebaf29974f9787961, user.id=50ecfbece5ba4fbf811003296fb76d4b) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:16.400548 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f9778-d241-434d-8fc7-8d55bd6b921f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:16.401963 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f9778-d241-434d-8fc7-8d55bd6b921f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:16.407495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 594/1200] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:15:16 2022] GET /identity/v3/role_assignments?user.id=50ecfbece5ba4fbf811003296fb76d4b&scope.project.id=984ff88e6a0147bebaf29974f9787961&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:26.253077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=54 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:26.404946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=64 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:10.977322 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0834fe3-afb7-4cf0-b4cb-430bb2277dc4 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:10.977903 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0834fe3-afb7-4cf0-b4cb-430bb2277dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:10.977903 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0834fe3-afb7-4cf0-b4cb-430bb2277dc4 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:10.979239 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 607/1201] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:10.985327 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7155b3-6073-41b2-b451-a59d2ab7ad62 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:10.986086 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7155b3-6073-41b2-b451-a59d2ab7ad62 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:10.986086 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7155b3-6073-41b2-b451-a59d2ab7ad62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:11.009671 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fb7155b3-6073-41b2-b451-a59d2ab7ad62 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:11.011725 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fb7155b3-6073-41b2-b451-a59d2ab7ad62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:11.107017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 595/1202] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:11.131346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0834fe3-afb7-4cf0-b4cb-430bb2277dc4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:11.250972 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0834fe3-afb7-4cf0-b4cb-430bb2277dc4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:11.296759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12caa1b-f971-4209-ab27-0a0097b48902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:11.297388 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12caa1b-f971-4209-ab27-0a0097b48902 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:11.297664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12caa1b-f971-4209-ab27-0a0097b48902 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:11.297934 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12caa1b-f971-4209-ab27-0a0097b48902 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:11.316481 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12caa1b-f971-4209-ab27-0a0097b48902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:11.318218 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12caa1b-f971-4209-ab27-0a0097b48902 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:11.340838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-b12caa1b-f971-4209-ab27-0a0097b48902 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:11.340838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:11.340838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:11.340838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:11.340838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:11.340838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:11.340838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:11.340838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:11.340838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:11.340838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:11.340838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:11.340838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:11.340838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:16:11.340838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:16:11.340838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:16:11.340838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:16:11.342349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:11.342349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:11.342349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:11.342349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:11.342349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:11.342349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:11.342349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:11.342349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:11.342349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:11.342349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:11.342349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:11.342349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:11.342349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:11.342349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:11.342349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:11.342349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:11.342349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:16:11.343323 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:16:11.343323 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:11.343323 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:16:11.343323 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 608/1203] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:16:11.349396 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7155b3-6073-41b2-b451-a59d2ab7ad62 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:11.491351 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7155b3-6073-41b2-b451-a59d2ab7ad62 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:11.529797 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:11.540103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c9b44-76fd-498c-846a-6c9faa3ad7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:11.540713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c9b44-76fd-498c-846a-6c9faa3ad7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:11.541033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c9b44-76fd-498c-846a-6c9faa3ad7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:11.541273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c9b44-76fd-498c-846a-6c9faa3ad7c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:11.541913 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c9b44-76fd-498c-846a-6c9faa3ad7c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:11.542229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c9b44-76fd-498c-846a-6c9faa3ad7c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:11.543615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c9b44-76fd-498c-846a-6c9faa3ad7c7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:11.562794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 596/1204] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:11.570050 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12caa1b-f971-4209-ab27-0a0097b48902 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:11.644552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:11.722386 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12caa1b-f971-4209-ab27-0a0097b48902 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:11.764057 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478d36a-1ace-4ea6-aa9a-da5a5e036790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:11.765073 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478d36a-1ace-4ea6-aa9a-da5a5e036790 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:11.765452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478d36a-1ace-4ea6-aa9a-da5a5e036790 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:11.765823 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478d36a-1ace-4ea6-aa9a-da5a5e036790 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:11.766796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478d36a-1ace-4ea6-aa9a-da5a5e036790 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:11.769064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478d36a-1ace-4ea6-aa9a-da5a5e036790 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:11.806372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 609/1205] 10.222.0.39 () {60 vars in 1132 bytes} [Mon Aug 8 02:16:11 2022] POST /identity/v3/users => generated 251 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:16:14.218133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b928a-8f86-4eed-8d72-75083115d094 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:14.218764 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b928a-8f86-4eed-8d72-75083115d094 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:14.218764 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b928a-8f86-4eed-8d72-75083115d094 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:14.220238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 597/1206] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:16:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:14.226698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac530e-992c-4770-b03b-346fbabe7342 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:14.227228 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac530e-992c-4770-b03b-346fbabe7342 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:14.229576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac530e-992c-4770-b03b-346fbabe7342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:14.248513 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-2eac530e-992c-4770-b03b-346fbabe7342 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:14.250057 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2eac530e-992c-4770-b03b-346fbabe7342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:14.341301 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 610/1207] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:14.366979 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b928a-8f86-4eed-8d72-75083115d094 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:14.502104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b928a-8f86-4eed-8d72-75083115d094 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:14.546381 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abd973-df4e-4f4a-9a94-e3608f598c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:14.546970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abd973-df4e-4f4a-9a94-e3608f598c67 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:14.547235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abd973-df4e-4f4a-9a94-e3608f598c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:14.547560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abd973-df4e-4f4a-9a94-e3608f598c67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:14.559192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67abd973-df4e-4f4a-9a94-e3608f598c67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:16:14.566034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-67abd973-df4e-4f4a-9a94-e3608f598c67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:16:14.566034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:14.566034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:14.566034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:14.566034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:14.566034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:14.566034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:14.566034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:14.566034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:14.566034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:14.566034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:14.566034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:14.566034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:16:14.566034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:16:14.566034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:16:14.566034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:16:14.567378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:16:14.567378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:16:14.567378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:16:14.567378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:16:14.567378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:14.567378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:14.567378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:14.567378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:14.567378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:14.567378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:14.567378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:14.567378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:14.567378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:14.567378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:14.567378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:14.567378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:14.568246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:14.568246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:14.568246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:14.568246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:14.568246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:16:14.568246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:16:14.568246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:16:14.568246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:16:14.568246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:16:14.568246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:16:14.568246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:16:14.568246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:14.568246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 598/1208] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:16:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:14.574025 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac530e-992c-4770-b03b-346fbabe7342 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:14.629185 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:14.708087 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac530e-992c-4770-b03b-346fbabe7342 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:14.752233 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d34b1-5055-4104-b39c-ce57e35431c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:14.752773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d34b1-5055-4104-b39c-ce57e35431c8 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:14.753053 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d34b1-5055-4104-b39c-ce57e35431c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:14.753325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d34b1-5055-4104-b39c-ce57e35431c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:14.753985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d34b1-5055-4104-b39c-ce57e35431c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:14.754311 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d34b1-5055-4104-b39c-ce57e35431c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:14.755798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d34b1-5055-4104-b39c-ce57e35431c8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:14.764618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 611/1209] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:16:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:14.771190 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abd973-df4e-4f4a-9a94-e3608f598c67 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:14.836427 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:14.900611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abd973-df4e-4f4a-9a94-e3608f598c67 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:14.942409 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0555111-4f36-48f9-9c33-8d8b352bbae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:14.943172 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0555111-4f36-48f9-9c33-8d8b352bbae1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:14.943479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0555111-4f36-48f9-9c33-8d8b352bbae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:14.943816 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0555111-4f36-48f9-9c33-8d8b352bbae1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:14.959383 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0555111-4f36-48f9-9c33-8d8b352bbae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:14.961214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0555111-4f36-48f9-9c33-8d8b352bbae1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:14.977752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e0555111-4f36-48f9-9c33-8d8b352bbae1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:14.977752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:14.977752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:14.977752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:14.977752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:14.977752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:14.977752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:14.977752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:14.977752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:14.977752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:14.977752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:14.977752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:14.977752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:16:14.977752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:16:14.977752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:16:14.977752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:16:14.979170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:14.979170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:14.979170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:14.979170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:14.979170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:14.979170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:14.979170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:14.979170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:14.979170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:14.979170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:14.979170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:14.979170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:14.979170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:14.979170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:14.979170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:14.979170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:14.979170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:16:14.980640 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:16:14.980640 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:14.980640 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:14.985815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 599/1210] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:14.987352 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d34b1-5055-4104-b39c-ce57e35431c8 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:15.114520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:15.118798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d34b1-5055-4104-b39c-ce57e35431c8 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:15.159364 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757ece4-4b20-4a84-adcc-9a28c4246437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:15.159968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757ece4-4b20-4a84-adcc-9a28c4246437 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:15.160241 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757ece4-4b20-4a84-adcc-9a28c4246437 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:15.160506 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757ece4-4b20-4a84-adcc-9a28c4246437 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:15.161142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757ece4-4b20-4a84-adcc-9a28c4246437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:15.161497 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757ece4-4b20-4a84-adcc-9a28c4246437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:15.162889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757ece4-4b20-4a84-adcc-9a28c4246437 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:15.176087 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 612/1211] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:15.181752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0555111-4f36-48f9-9c33-8d8b352bbae1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:15.291225 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0555111-4f36-48f9-9c33-8d8b352bbae1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:15.292953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:15.331194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d2155-87e2-4803-8495-45e3b2f5d258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:15.331891 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d2155-87e2-4803-8495-45e3b2f5d258 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:15.332171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d2155-87e2-4803-8495-45e3b2f5d258 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:15.332446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d2155-87e2-4803-8495-45e3b2f5d258 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:15.341503 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d2155-87e2-4803-8495-45e3b2f5d258 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:15.343037 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d2155-87e2-4803-8495-45e3b2f5d258 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:15.355337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5d2d2155-87e2-4803-8495-45e3b2f5d258 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:16:15.355337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:15.355337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:15.355337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:15.355337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:15.355337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:15.355337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:15.355337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:15.355337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:15.355337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:15.355337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:15.355337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:15.355337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:16:15.355337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:16:15.355337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:16:15.355337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:16:15.359033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:15.359033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:15.359033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:16:15.359033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:16:15.359033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:16:15.359033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:16:15.359033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:15.359033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:15.359033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:15.359033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:15.359033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:15.359033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:15.359033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:15.359033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:15.359033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:15.359033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:15.359033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:15.359033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:15.360211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:15.360211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:15.360211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:16:15.360211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:16:15.360211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:16:15.360211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:16:15.360211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:16:15.360211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:16:15.360211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:16:15.360211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:15.360211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 600/1212] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:16:15.363503 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757ece4-4b20-4a84-adcc-9a28c4246437 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:15.489453 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757ece4-4b20-4a84-adcc-9a28c4246437 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:15.553267 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7a4ff-c413-4685-b059-69e8aaec92b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:15.553923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7a4ff-c413-4685-b059-69e8aaec92b0 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:15.554200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7a4ff-c413-4685-b059-69e8aaec92b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:15.554491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7a4ff-c413-4685-b059-69e8aaec92b0 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:15.555222 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7a4ff-c413-4685-b059-69e8aaec92b0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:15.555526 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7a4ff-c413-4685-b059-69e8aaec92b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:15.556941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7a4ff-c413-4685-b059-69e8aaec92b0 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:15.583326 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 613/1213] 10.222.0.39 () {56 vars in 1143 bytes} [Mon Aug 8 02:16:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:15.586677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d2155-87e2-4803-8495-45e3b2f5d258 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:15.711669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d2155-87e2-4803-8495-45e3b2f5d258 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:15.760540 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0ed4b-5c41-4171-a233-7de9b4514b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:15.761158 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0ed4b-5c41-4171-a233-7de9b4514b04 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:15.761435 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0ed4b-5c41-4171-a233-7de9b4514b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:15.761699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0ed4b-5c41-4171-a233-7de9b4514b04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:15.793555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:15.847359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0ed4b-5c41-4171-a233-7de9b4514b04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:15.848868 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0ed4b-5c41-4171-a233-7de9b4514b04 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:15.977639 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 601/1214] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:16:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:16:15.983990 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7a4ff-c413-4685-b059-69e8aaec92b0 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:16.004496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:16.097217 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7a4ff-c413-4685-b059-69e8aaec92b0 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:16.136750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca951d2b-2ac0-42cf-86a4-63f45ac09644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:16.137519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca951d2b-2ac0-42cf-86a4-63f45ac09644 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:16.137846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca951d2b-2ac0-42cf-86a4-63f45ac09644 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:16.138117 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca951d2b-2ac0-42cf-86a4-63f45ac09644 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:16.149734 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca951d2b-2ac0-42cf-86a4-63f45ac09644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:16.151253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca951d2b-2ac0-42cf-86a4-63f45ac09644 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:16.167534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-ca951d2b-2ac0-42cf-86a4-63f45ac09644 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:16.167534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:16.167534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:16.167534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:16.167534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:16.167534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:16.167534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:16.167534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:16.167534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:16.167534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:16.167534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:16.167534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:16.167534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:16:16.167534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:16:16.167534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:16:16.167534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:16:16.168538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:16.168538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:16.168538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:16.168538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:16.168538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:16.168538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:16.168538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:16.168538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:16.168538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:16.168538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:16.168538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:16.168538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:16.168538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:16.168538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:16.168538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:16.168538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:16.168538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:16:16.169431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:16:16.169431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:16.169431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:16:16.169431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 614/1215] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:16:16.174302 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0ed4b-5c41-4171-a233-7de9b4514b04 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:16.252947 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:16.296074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0ed4b-5c41-4171-a233-7de9b4514b04 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:16.333075 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ac31c-3cf4-429e-a65e-2218c1d4a75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:16.333781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ac31c-3cf4-429e-a65e-2218c1d4a75f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:16.334055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ac31c-3cf4-429e-a65e-2218c1d4a75f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:16.334362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ac31c-3cf4-429e-a65e-2218c1d4a75f None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:16.335028 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ac31c-3cf4-429e-a65e-2218c1d4a75f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:16.335299 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ac31c-3cf4-429e-a65e-2218c1d4a75f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:16.336675 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ac31c-3cf4-429e-a65e-2218c1d4a75f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:16.348239 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 602/1216] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:16:16.354964 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca951d2b-2ac0-42cf-86a4-63f45ac09644 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:16.430667 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:16.466104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca951d2b-2ac0-42cf-86a4-63f45ac09644 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:16.507414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a965d9-2f01-4b31-b9ac-d7d87ab31c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:16.508449 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a965d9-2f01-4b31-b9ac-d7d87ab31c47 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:16.508729 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a965d9-2f01-4b31-b9ac-d7d87ab31c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:16.509028 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a965d9-2f01-4b31-b9ac-d7d87ab31c47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:16.516188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a965d9-2f01-4b31-b9ac-d7d87ab31c47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:16.518269 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a965d9-2f01-4b31-b9ac-d7d87ab31c47 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:16.529540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-68a965d9-2f01-4b31-b9ac-d7d87ab31c47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:16:16.529540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:16.529540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:16.529540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:16.529540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:16.529540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:16.529540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:16.529540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:16.529540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:16.529540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:16.529540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:16.529540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:16.529540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:16:16.529540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:16:16.529540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:16:16.530511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:16:16.530511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:16.530511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:16.530511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:16.530511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:16.530511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:16.530511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:16.530511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:16.530511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:16.530511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:16.530511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:16.530511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:16.530511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:16.530511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:16.530511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:16.530511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:16.530511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:16.531424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:16:16.531424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:16:16.531424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:16:16.531424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:16:16.531424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:16:16.531424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:16:16.531424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:16:16.531424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:16:16.531424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 615/1217] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:16:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:16.536630 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ac31c-3cf4-429e-a65e-2218c1d4a75f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:16.643603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ac31c-3cf4-429e-a65e-2218c1d4a75f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:16.665619 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:16.679699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db0902-b1d7-46ba-8a9a-5c93569e687c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:16.680626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db0902-b1d7-46ba-8a9a-5c93569e687c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:16.681074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db0902-b1d7-46ba-8a9a-5c93569e687c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:16.681528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db0902-b1d7-46ba-8a9a-5c93569e687c None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:16.682441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db0902-b1d7-46ba-8a9a-5c93569e687c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:16.682922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db0902-b1d7-46ba-8a9a-5c93569e687c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:16.684569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db0902-b1d7-46ba-8a9a-5c93569e687c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:16.699560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 603/1218] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:16:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:16:16.705873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a965d9-2f01-4b31-b9ac-d7d87ab31c47 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:16.826658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a965d9-2f01-4b31-b9ac-d7d87ab31c47 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:16.860015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:16.869662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587af94a-7a9c-48d2-9713-08c4ae0c8662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:16.870546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587af94a-7a9c-48d2-9713-08c4ae0c8662 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:16.870974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587af94a-7a9c-48d2-9713-08c4ae0c8662 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:16.871403 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587af94a-7a9c-48d2-9713-08c4ae0c8662 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:16.872328 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587af94a-7a9c-48d2-9713-08c4ae0c8662 None admin] RBAC: Adding query filter params (role.id=838b081e674f4c0ab4a33bcf8e614dc5, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=f62745b92a864480a9438de581e1d05e) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:16.872810 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587af94a-7a9c-48d2-9713-08c4ae0c8662 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:16.874696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587af94a-7a9c-48d2-9713-08c4ae0c8662 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:16.884423 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 616/1219] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:16:16 2022] GET /identity/v3/role_assignments?user.id=f62745b92a864480a9438de581e1d05e&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=838b081e674f4c0ab4a33bcf8e614dc5 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:19.023753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f83ca9-6529-4f8a-90e1-1d2ea1f9478a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:19.024372 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f83ca9-6529-4f8a-90e1-1d2ea1f9478a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:19.024372 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f83ca9-6529-4f8a-90e1-1d2ea1f9478a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:19.025759 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 604/1220] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:19.031754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664563f-14bf-4bba-abfc-db3aa2853fe2 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:19.032293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664563f-14bf-4bba-abfc-db3aa2853fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:19.032400 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664563f-14bf-4bba-abfc-db3aa2853fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:19.051799 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-9664563f-14bf-4bba-abfc-db3aa2853fe2 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:19.052987 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9664563f-14bf-4bba-abfc-db3aa2853fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:19.128736 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 617/1221] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:19.152830 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f83ca9-6529-4f8a-90e1-1d2ea1f9478a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:19.262878 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f83ca9-6529-4f8a-90e1-1d2ea1f9478a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:19.300697 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d509d5-a48c-4f3d-bae5-5a672958c028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:19.301285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d509d5-a48c-4f3d-bae5-5a672958c028 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:19.301580 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d509d5-a48c-4f3d-bae5-5a672958c028 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:19.301846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d509d5-a48c-4f3d-bae5-5a672958c028 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:19.312749 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d509d5-a48c-4f3d-bae5-5a672958c028 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:16:19.317953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-42d509d5-a48c-4f3d-bae5-5a672958c028 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:16:19.317953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:19.317953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:19.317953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:19.317953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:19.317953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:19.317953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:19.317953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:19.317953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:19.317953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:19.317953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:19.317953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:19.317953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:16:19.317953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:16:19.317953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:16:19.317953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:16:19.319321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:16:19.319321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:16:19.319321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:16:19.319321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:16:19.319321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:19.319321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:19.319321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:19.319321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:19.319321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:19.319321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:19.319321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:19.319321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:19.319321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:19.319321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:19.319321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:19.319321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:19.320235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:19.320235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:19.320235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:19.320235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:19.320235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:16:19.320235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:16:19.320235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:16:19.320235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:16:19.320235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:16:19.320235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:16:19.320235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:16:19.320235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:19.320235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 605/1222] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:16:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:19.325347 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664563f-14bf-4bba-abfc-db3aa2853fe2 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:19.429369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664563f-14bf-4bba-abfc-db3aa2853fe2 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:19.454399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:19.467284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7aeb89-33b9-4b8a-80a3-4d31763b1b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:19.467924 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7aeb89-33b9-4b8a-80a3-4d31763b1b01 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:19.468234 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7aeb89-33b9-4b8a-80a3-4d31763b1b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:19.468494 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7aeb89-33b9-4b8a-80a3-4d31763b1b01 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:19.469199 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7aeb89-33b9-4b8a-80a3-4d31763b1b01 None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:19.469501 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7aeb89-33b9-4b8a-80a3-4d31763b1b01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:19.470932 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7aeb89-33b9-4b8a-80a3-4d31763b1b01 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:19.479522 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 618/1223] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:16:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:19.485690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d509d5-a48c-4f3d-bae5-5a672958c028 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:19.530429 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:19.591805 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d509d5-a48c-4f3d-bae5-5a672958c028 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:19.631952 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22997eac-7a40-4673-a747-066ed6204600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:19.632584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22997eac-7a40-4673-a747-066ed6204600 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:19.632864 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22997eac-7a40-4673-a747-066ed6204600 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:19.633131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22997eac-7a40-4673-a747-066ed6204600 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:19.643590 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22997eac-7a40-4673-a747-066ed6204600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:19.645046 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22997eac-7a40-4673-a747-066ed6204600 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:19.658768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-22997eac-7a40-4673-a747-066ed6204600 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:19.658768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:19.658768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:19.658768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:19.658768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:19.658768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:19.658768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:19.658768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:19.658768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:19.658768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:19.658768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:19.658768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:19.658768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:16:19.658768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:16:19.658768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:16:19.658768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:16:19.659850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:19.659850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:19.659850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:19.659850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:19.659850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:19.659850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:19.659850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:19.659850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:19.659850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:19.659850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:19.659850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:19.659850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:19.659850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:19.659850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:19.659850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:19.659850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:19.659850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:16:19.661077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:16:19.661077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:19.661077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:19.661077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 606/1224] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:16:19.666207 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7aeb89-33b9-4b8a-80a3-4d31763b1b01 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:19.775990 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7aeb89-33b9-4b8a-80a3-4d31763b1b01 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:19.811300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f72a4-df9e-499a-a229-0c38f5f1d47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:19.811930 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f72a4-df9e-499a-a229-0c38f5f1d47a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:19.812240 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f72a4-df9e-499a-a229-0c38f5f1d47a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:19.812547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f72a4-df9e-499a-a229-0c38f5f1d47a None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:19.813183 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f72a4-df9e-499a-a229-0c38f5f1d47a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:19.813482 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f72a4-df9e-499a-a229-0c38f5f1d47a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:19.815475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f72a4-df9e-499a-a229-0c38f5f1d47a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:19.827136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 619/1225] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:19.832990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22997eac-7a40-4673-a747-066ed6204600 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:19.929802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:19.937981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22997eac-7a40-4673-a747-066ed6204600 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:19.975255 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3640eb-a51d-4648-ae0b-1802c3bf01cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:19.975967 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3640eb-a51d-4648-ae0b-1802c3bf01cc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:19.976244 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3640eb-a51d-4648-ae0b-1802c3bf01cc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:19.976510 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3640eb-a51d-4648-ae0b-1802c3bf01cc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:19.984041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3640eb-a51d-4648-ae0b-1802c3bf01cc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:19.985485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3640eb-a51d-4648-ae0b-1802c3bf01cc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:19.998598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5d3640eb-a51d-4648-ae0b-1802c3bf01cc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:16:19.998598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:19.998598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:19.998598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:19.998598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:19.998598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:19.998598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:19.998598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:19.998598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:19.998598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:19.998598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:19.998598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:19.998598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:16:19.998598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:16:19.998598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:16:19.998598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:16:19.999606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:19.999606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:19.999606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:16:19.999606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:16:19.999606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:16:19.999606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:16:19.999606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:19.999606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:19.999606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:19.999606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:19.999606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:19.999606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:19.999606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:19.999606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:19.999606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:19.999606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:19.999606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:19.999606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:20.000748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:20.000748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:20.000748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:16:20.000748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:16:20.000748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:16:20.000748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:16:20.000748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:16:20.000748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:16:20.000748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:16:20.000748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:20.000748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 607/1226] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:20.006448 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f72a4-df9e-499a-a229-0c38f5f1d47a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:20.059016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:20.122417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f72a4-df9e-499a-a229-0c38f5f1d47a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:20.172206 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5ed66-dc1e-4cb3-9af7-1df40e5ffe11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:20.173186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5ed66-dc1e-4cb3-9af7-1df40e5ffe11 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:20.174213 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5ed66-dc1e-4cb3-9af7-1df40e5ffe11 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:20.174627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5ed66-dc1e-4cb3-9af7-1df40e5ffe11 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:20.175580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5ed66-dc1e-4cb3-9af7-1df40e5ffe11 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:20.175982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5ed66-dc1e-4cb3-9af7-1df40e5ffe11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:20.178082 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5ed66-dc1e-4cb3-9af7-1df40e5ffe11 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:20.197781 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 620/1227] 10.222.0.39 () {56 vars in 1143 bytes} [Mon Aug 8 02:16:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:16:20.203738 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3640eb-a51d-4648-ae0b-1802c3bf01cc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:20.313531 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3640eb-a51d-4648-ae0b-1802c3bf01cc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:20.355410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111a0ef-5bd9-4f25-b0bd-5e15ed51c5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:20.356102 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111a0ef-5bd9-4f25-b0bd-5e15ed51c5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:20.356365 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111a0ef-5bd9-4f25-b0bd-5e15ed51c5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:20.356657 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111a0ef-5bd9-4f25-b0bd-5e15ed51c5e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:20.368592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111a0ef-5bd9-4f25-b0bd-5e15ed51c5e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:20.370130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111a0ef-5bd9-4f25-b0bd-5e15ed51c5e8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:20.388618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a111a0ef-5bd9-4f25-b0bd-5e15ed51c5e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:20.388618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:20.388618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:20.388618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:20.388618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:20.388618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:20.388618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:20.388618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:20.388618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:20.388618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:20.388618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:20.388618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:20.388618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:16:20.388618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:16:20.388618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:16:20.388618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:16:20.389761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:20.389761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:20.389761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:20.389761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:20.389761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:20.389761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:20.389761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:20.389761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:20.389761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:20.389761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:20.389761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:20.389761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:20.389761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:20.389761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:20.389761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:20.389761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:20.389761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:16:20.390829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:16:20.390829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:20.390829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:20.393441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 608/1228] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:16:20.402660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5ed66-dc1e-4cb3-9af7-1df40e5ffe11 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:20.532116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5ed66-dc1e-4cb3-9af7-1df40e5ffe11 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:20.535301 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:20.572979 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d5482-58ec-483e-891b-b66c6c5d019f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:20.573590 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d5482-58ec-483e-891b-b66c6c5d019f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:20.573988 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d5482-58ec-483e-891b-b66c6c5d019f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:20.574382 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d5482-58ec-483e-891b-b66c6c5d019f None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:20.575244 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d5482-58ec-483e-891b-b66c6c5d019f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:20.575659 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d5482-58ec-483e-891b-b66c6c5d019f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:20.577287 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d5482-58ec-483e-891b-b66c6c5d019f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:20.593675 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 621/1229] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:20.600538 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111a0ef-5bd9-4f25-b0bd-5e15ed51c5e8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:20.611523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:20.712847 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111a0ef-5bd9-4f25-b0bd-5e15ed51c5e8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:20.745414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d604f1-4454-4e85-974b-806d14bde581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:20.746027 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d604f1-4454-4e85-974b-806d14bde581 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:20.746327 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d604f1-4454-4e85-974b-806d14bde581 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:20.746594 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d604f1-4454-4e85-974b-806d14bde581 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:20.753631 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d604f1-4454-4e85-974b-806d14bde581 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:20.755100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d604f1-4454-4e85-974b-806d14bde581 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:20.768160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e7d604f1-4454-4e85-974b-806d14bde581 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:16:20.768160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:20.768160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:20.768160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:20.768160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:20.768160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:20.768160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:20.768160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:20.768160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:20.768160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:20.768160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:20.768160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:20.768160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:16:20.768160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:16:20.768160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:16:20.769264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:16:20.769264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:20.769264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:20.769264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:20.769264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:20.769264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:20.769264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:20.769264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:20.769264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:20.769264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:20.769264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:20.769264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:20.769264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:20.769264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:20.769264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:20.769264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:20.769264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:20.774872 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:16:20.774872 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:16:20.774872 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:16:20.774872 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:16:20.774872 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:16:20.774872 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:16:20.774872 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:16:20.774872 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:20.774872 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 609/1230] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:16:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:20.775390 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d5482-58ec-483e-891b-b66c6c5d019f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:20.877054 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d5482-58ec-483e-891b-b66c6c5d019f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:20.910550 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3add6b-2b58-4514-a803-36d2a1bc9aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:20.911153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3add6b-2b58-4514-a803-36d2a1bc9aec None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:20.911426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3add6b-2b58-4514-a803-36d2a1bc9aec None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:20.911691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3add6b-2b58-4514-a803-36d2a1bc9aec None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:20.912370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3add6b-2b58-4514-a803-36d2a1bc9aec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:20.912669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3add6b-2b58-4514-a803-36d2a1bc9aec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:20.914080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3add6b-2b58-4514-a803-36d2a1bc9aec None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:20.926306 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 622/1231] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:16:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:16:20.932565 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d604f1-4454-4e85-974b-806d14bde581 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:21.004263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:21.043061 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d604f1-4454-4e85-974b-806d14bde581 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:21.082576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d828f3-5627-4167-9d03-1d0d1ce640ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:21.083344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d828f3-5627-4167-9d03-1d0d1ce640ad None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:21.083693 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d828f3-5627-4167-9d03-1d0d1ce640ad None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:21.084024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d828f3-5627-4167-9d03-1d0d1ce640ad None admin] PATH_INFO: `/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/f62745b92a864480a9438de581e1d05e/roles/838b081e674f4c0ab4a33bcf8e614dc5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:21.101389 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d828f3-5627-4167-9d03-1d0d1ce640ad None admin] RBAC: Authorizing `identity:create_grant(project_id=7d474b61f1fe4f17aacb7a3665315c79, user_id=f62745b92a864480a9438de581e1d05e, role_id=838b081e674f4c0ab4a33bcf8e614dc5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:21.103004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d828f3-5627-4167-9d03-1d0d1ce640ad None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:21.122013 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 610/1232] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:16:20 2022] PUT /identity/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/f62745b92a864480a9438de581e1d05e/roles/838b081e674f4c0ab4a33bcf8e614dc5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:16:23.245900 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affe6e6-895b-42cc-bff3-6d898dcb56ba None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:23.246482 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affe6e6-895b-42cc-bff3-6d898dcb56ba None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:23.246482 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affe6e6-895b-42cc-bff3-6d898dcb56ba None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:23.247796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 623/1233] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:23.253129 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627d259-3330-41e8-877a-f1d9eec4dd2d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:23.253610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627d259-3330-41e8-877a-f1d9eec4dd2d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:23.253707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627d259-3330-41e8-877a-f1d9eec4dd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:23.271846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7627d259-3330-41e8-877a-f1d9eec4dd2d None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:23.273028 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7627d259-3330-41e8-877a-f1d9eec4dd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:23.344039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 611/1234] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:23.368825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affe6e6-895b-42cc-bff3-6d898dcb56ba None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:23.397405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:23.480459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affe6e6-895b-42cc-bff3-6d898dcb56ba None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:23.516349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac7042-d6fe-4ec4-9a1f-5f0c7eb4008e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:23.517473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac7042-d6fe-4ec4-9a1f-5f0c7eb4008e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:23.517746 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac7042-d6fe-4ec4-9a1f-5f0c7eb4008e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:23.518013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac7042-d6fe-4ec4-9a1f-5f0c7eb4008e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:23.528867 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac7042-d6fe-4ec4-9a1f-5f0c7eb4008e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:16:23.534696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-10ac7042-d6fe-4ec4-9a1f-5f0c7eb4008e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:16:23.534696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:23.534696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:23.534696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:23.534696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:23.534696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:23.534696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:23.534696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:23.534696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:23.534696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:23.534696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:23.534696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:23.534696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:16:23.534696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:16:23.534696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:16:23.534696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:16:23.535835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:16:23.535835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:16:23.535835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:16:23.535835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:16:23.535835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:23.535835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:23.535835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:23.535835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:23.535835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:23.535835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:23.535835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:23.535835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:23.535835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:23.535835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:23.535835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:23.535835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:23.536759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:23.536759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:23.536759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:23.536759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:23.536759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:16:23.536759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:16:23.536759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:16:23.536759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:16:23.536759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:16:23.536759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:16:23.536759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:16:23.536759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:16:23.541065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 624/1235] 10.222.0.39 () {56 vars in 1102 bytes} [Mon Aug 8 02:16:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:23.542174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627d259-3330-41e8-877a-f1d9eec4dd2d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:23.592537 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:23.652942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627d259-3330-41e8-877a-f1d9eec4dd2d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:23.689086 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb8946-1d78-452a-93cf-4c13e88e2ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:23.689751 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb8946-1d78-452a-93cf-4c13e88e2ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:23.690030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb8946-1d78-452a-93cf-4c13e88e2ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:23.690332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb8946-1d78-452a-93cf-4c13e88e2ba2 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:23.690993 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb8946-1d78-452a-93cf-4c13e88e2ba2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:23.691296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb8946-1d78-452a-93cf-4c13e88e2ba2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:23.692701 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb8946-1d78-452a-93cf-4c13e88e2ba2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:23.702108 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 612/1236] 10.222.0.39 () {56 vars in 1103 bytes} [Mon Aug 8 02:16:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:16:23.709079 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac7042-d6fe-4ec4-9a1f-5f0c7eb4008e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:23.799334 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:23.816512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac7042-d6fe-4ec4-9a1f-5f0c7eb4008e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:23.850748 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5407bc9-d869-418e-8511-a1743717cd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:23.851364 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5407bc9-d869-418e-8511-a1743717cd46 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:23.851637 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5407bc9-d869-418e-8511-a1743717cd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:23.851946 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5407bc9-d869-418e-8511-a1743717cd46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:23.862348 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5407bc9-d869-418e-8511-a1743717cd46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:23.863791 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5407bc9-d869-418e-8511-a1743717cd46 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:23.878409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-f5407bc9-d869-418e-8511-a1743717cd46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:23.878409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:23.878409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:23.878409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:23.878409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:23.878409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:23.878409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:23.878409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:23.878409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:23.878409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:23.878409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:23.878409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:23.878409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:16:23.878409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:16:23.878409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:16:23.878409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:16:23.879487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:23.879487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:23.879487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:23.879487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:23.879487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:23.879487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:23.879487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:23.879487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:23.879487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:23.879487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:23.879487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:23.879487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:23.879487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:23.879487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:23.879487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:23.879487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:23.879487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:16:23.880370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:16:23.880370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:23.880370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:16:23.880370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 625/1237] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:23.885550 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb8946-1d78-452a-93cf-4c13e88e2ba2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:23.993589 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb8946-1d78-452a-93cf-4c13e88e2ba2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:24.006234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:24.033269 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13887035-720c-4e79-aaca-cbb04aedc8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:24.033912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13887035-720c-4e79-aaca-cbb04aedc8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:24.034202 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13887035-720c-4e79-aaca-cbb04aedc8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:24.034511 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13887035-720c-4e79-aaca-cbb04aedc8a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:24.035154 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13887035-720c-4e79-aaca-cbb04aedc8a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:24.035482 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13887035-720c-4e79-aaca-cbb04aedc8a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:24.037053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13887035-720c-4e79-aaca-cbb04aedc8a5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:24.049900 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 613/1238] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:24.055845 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5407bc9-d869-418e-8511-a1743717cd46 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:24.177069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5407bc9-d869-418e-8511-a1743717cd46 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:24.215640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d48784-b1e3-4553-8b46-d1683a005b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:24.216304 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d48784-b1e3-4553-8b46-d1683a005b82 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:24.216622 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d48784-b1e3-4553-8b46-d1683a005b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:24.216923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d48784-b1e3-4553-8b46-d1683a005b82 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:24.219837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:24.226046 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d48784-b1e3-4553-8b46-d1683a005b82 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:24.227643 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d48784-b1e3-4553-8b46-d1683a005b82 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:24.240077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-60d48784-b1e3-4553-8b46-d1683a005b82 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:16:24.240077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:24.240077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:24.240077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:24.240077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:24.240077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:24.240077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:24.240077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:24.240077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:24.240077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:24.240077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:24.240077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:24.240077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:16:24.240077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:16:24.240077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:16:24.240077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:16:24.241139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:24.241139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:24.241139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:16:24.241139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:16:24.241139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:16:24.241139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:16:24.241139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:24.241139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:24.241139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:24.241139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:24.241139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:24.241139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:24.241139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:24.241139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:24.241139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:24.241139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:24.241139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:24.241139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:24.242078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:24.242078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:24.242078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:16:24.242078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:16:24.242078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:16:24.242078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:16:24.242078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:16:24.242078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:16:24.242078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:16:24.242078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:16:24.242078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 626/1239] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:24.246997 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13887035-720c-4e79-aaca-cbb04aedc8a5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:24.364645 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13887035-720c-4e79-aaca-cbb04aedc8a5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:24.402321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae859ae-fa1c-452f-8d84-27f0f5f3e09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:24.403016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae859ae-fa1c-452f-8d84-27f0f5f3e09f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:24.403301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae859ae-fa1c-452f-8d84-27f0f5f3e09f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:24.403570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae859ae-fa1c-452f-8d84-27f0f5f3e09f None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:24.404513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae859ae-fa1c-452f-8d84-27f0f5f3e09f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:24.404857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae859ae-fa1c-452f-8d84-27f0f5f3e09f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:24.406427 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae859ae-fa1c-452f-8d84-27f0f5f3e09f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:24.426068 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 614/1240] 10.222.0.39 () {56 vars in 1143 bytes} [Mon Aug 8 02:16:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:24.432117 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d48784-b1e3-4553-8b46-d1683a005b82 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:24.549440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d48784-b1e3-4553-8b46-d1683a005b82 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:24.586141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2fdf1-83eb-4131-b0b7-c163b1a5625d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:24.586772 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2fdf1-83eb-4131-b0b7-c163b1a5625d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:24.587051 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2fdf1-83eb-4131-b0b7-c163b1a5625d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:24.587322 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2fdf1-83eb-4131-b0b7-c163b1a5625d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:24.653713 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:24.662414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2fdf1-83eb-4131-b0b7-c163b1a5625d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:24.664252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2fdf1-83eb-4131-b0b7-c163b1a5625d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:24.770272 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 627/1241] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:16:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:16:24.776320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae859ae-fa1c-452f-8d84-27f0f5f3e09f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:24.804828 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:24.886605 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae859ae-fa1c-452f-8d84-27f0f5f3e09f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:24.923823 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede38c5-48f6-4770-b89a-308f660899e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:24.924372 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede38c5-48f6-4770-b89a-308f660899e4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:24.924647 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede38c5-48f6-4770-b89a-308f660899e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:24.925007 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede38c5-48f6-4770-b89a-308f660899e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:24.935518 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede38c5-48f6-4770-b89a-308f660899e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:24.936965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede38c5-48f6-4770-b89a-308f660899e4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:24.952418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-aede38c5-48f6-4770-b89a-308f660899e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:24.952418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:24.952418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:24.952418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:24.952418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:24.952418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:24.952418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:24.952418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:24.952418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:24.952418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:24.952418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:24.952418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:24.952418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:16:24.952418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:16:24.952418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:16:24.952418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:16:24.953425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:24.953425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:24.953425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:24.953425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:24.953425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:24.953425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:24.953425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:24.953425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:24.953425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:24.953425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:24.953425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:24.953425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:24.953425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:24.953425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:24.953425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:24.953425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:24.953425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:16:24.955267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:16:24.955267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:24.955267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:24.955267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 615/1242] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:24.960509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2fdf1-83eb-4131-b0b7-c163b1a5625d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:25.063081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:25.070981 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2fdf1-83eb-4131-b0b7-c163b1a5625d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:25.117261 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3914d-b23b-4efb-a874-66a3a5f0a919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:25.117985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3914d-b23b-4efb-a874-66a3a5f0a919 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:25.118340 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3914d-b23b-4efb-a874-66a3a5f0a919 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:25.118605 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3914d-b23b-4efb-a874-66a3a5f0a919 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:25.119255 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3914d-b23b-4efb-a874-66a3a5f0a919 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:25.119558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3914d-b23b-4efb-a874-66a3a5f0a919 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:25.121040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3914d-b23b-4efb-a874-66a3a5f0a919 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:25.134815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 628/1243] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:25.141126 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede38c5-48f6-4770-b89a-308f660899e4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:25.225510 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:25.247649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede38c5-48f6-4770-b89a-308f660899e4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:25.285347 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05069d47-2c2f-4889-9e8b-a42f60a09426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:25.285963 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05069d47-2c2f-4889-9e8b-a42f60a09426 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:25.286238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05069d47-2c2f-4889-9e8b-a42f60a09426 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:25.286532 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05069d47-2c2f-4889-9e8b-a42f60a09426 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:25.293152 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05069d47-2c2f-4889-9e8b-a42f60a09426 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:25.294623 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05069d47-2c2f-4889-9e8b-a42f60a09426 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:25.306526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-05069d47-2c2f-4889-9e8b-a42f60a09426 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:16:25.306526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:25.306526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:25.306526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:25.306526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:25.306526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:25.306526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:25.306526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:25.306526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:25.306526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:25.306526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:25.306526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:25.306526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:16:25.306526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:16:25.306526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:16:25.307603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:16:25.307603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:25.307603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:25.307603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:25.307603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:25.307603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:25.307603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:25.307603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:25.307603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:25.307603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:25.307603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:25.307603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:25.307603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:25.307603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:25.307603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:25.307603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:25.307603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:25.308524 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:16:25.308524 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:16:25.308524 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:16:25.308524 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:16:25.308524 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:16:25.308524 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:16:25.308524 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:16:25.308524 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:25.308524 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 616/1244] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:16:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:25.314594 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3914d-b23b-4efb-a874-66a3a5f0a919 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:25.422103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3914d-b23b-4efb-a874-66a3a5f0a919 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:25.458883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3df9a-0ce1-4ec0-a0f8-01aaf1adfaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:25.459748 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3df9a-0ce1-4ec0-a0f8-01aaf1adfaed None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:25.459902 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3df9a-0ce1-4ec0-a0f8-01aaf1adfaed None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:25.460061 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3df9a-0ce1-4ec0-a0f8-01aaf1adfaed None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:25.460742 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3df9a-0ce1-4ec0-a0f8-01aaf1adfaed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:25.461051 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3df9a-0ce1-4ec0-a0f8-01aaf1adfaed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:25.462864 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3df9a-0ce1-4ec0-a0f8-01aaf1adfaed None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:25.478976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 629/1245] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:16:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:25.485475 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05069d47-2c2f-4889-9e8b-a42f60a09426 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:25.593407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05069d47-2c2f-4889-9e8b-a42f60a09426 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:25.638065 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b3e52-9de7-4b1d-a78d-5c721bf02536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:25.638690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b3e52-9de7-4b1d-a78d-5c721bf02536 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:25.638969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b3e52-9de7-4b1d-a78d-5c721bf02536 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:25.639236 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b3e52-9de7-4b1d-a78d-5c721bf02536 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:25.639959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b3e52-9de7-4b1d-a78d-5c721bf02536 None admin] RBAC: Adding query filter params (role.id=838b081e674f4c0ab4a33bcf8e614dc5, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=f62745b92a864480a9438de581e1d05e) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:25.640354 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b3e52-9de7-4b1d-a78d-5c721bf02536 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:25.641949 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b3e52-9de7-4b1d-a78d-5c721bf02536 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:25.649557 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 617/1246] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:16:25 2022] GET /identity/v3/role_assignments?user.id=f62745b92a864480a9438de581e1d05e&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=838b081e674f4c0ab4a33bcf8e614dc5 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:27.710985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90f423-5e64-41e0-a521-dae060d9055e None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:27.711517 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90f423-5e64-41e0-a521-dae060d9055e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:27.711517 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90f423-5e64-41e0-a521-dae060d9055e None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:27.713569 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 630/1247] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:16:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:27.719621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa17fe4-3afa-49b4-be95-52034e5f6c7a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:27.720605 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa17fe4-3afa-49b4-be95-52034e5f6c7a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:27.720756 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa17fe4-3afa-49b4-be95-52034e5f6c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:27.727709 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:27.741737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4aa17fe4-3afa-49b4-be95-52034e5f6c7a None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:27.743007 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4aa17fe4-3afa-49b4-be95-52034e5f6c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:27.816637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 618/1248] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:27.840109 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90f423-5e64-41e0-a521-dae060d9055e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:27.860490 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:27.966183 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90f423-5e64-41e0-a521-dae060d9055e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:28.001203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3566a17-decf-474c-b7ad-4f0c8e08e2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:28.001832 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3566a17-decf-474c-b7ad-4f0c8e08e2ee None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:28.002108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3566a17-decf-474c-b7ad-4f0c8e08e2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:28.002411 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3566a17-decf-474c-b7ad-4f0c8e08e2ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:28.013305 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3566a17-decf-474c-b7ad-4f0c8e08e2ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:16:28.018665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-d3566a17-decf-474c-b7ad-4f0c8e08e2ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:16:28.018665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:28.018665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:28.018665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:28.018665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:28.018665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:28.018665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:28.018665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:28.018665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:28.018665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:28.018665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:28.018665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:28.018665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:16:28.018665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:16:28.018665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:16:28.018665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:16:28.019787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:16:28.019787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:16:28.019787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:16:28.019787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:16:28.019787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:28.019787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:28.019787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:28.019787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:28.019787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:28.019787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:28.019787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:28.019787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:28.019787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:28.019787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:28.019787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:28.019787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:28.025911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:28.025911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:28.025911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:28.025911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:28.025911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:16:28.025911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:16:28.025911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:16:28.025911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:16:28.025911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:16:28.025911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:16:28.025911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:16:28.025911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:16:28.025911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 631/1249] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:16:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:28.026663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa17fe4-3afa-49b4-be95-52034e5f6c7a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:28.145315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa17fe4-3afa-49b4-be95-52034e5f6c7a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:28.180969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:28.188020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7943c3-22e8-4f34-bc42-0ce39fc0e8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:28.188630 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7943c3-22e8-4f34-bc42-0ce39fc0e8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:28.188952 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7943c3-22e8-4f34-bc42-0ce39fc0e8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:28.189302 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7943c3-22e8-4f34-bc42-0ce39fc0e8c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:28.190177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7943c3-22e8-4f34-bc42-0ce39fc0e8c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:28.190596 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7943c3-22e8-4f34-bc42-0ce39fc0e8c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:28.192240 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7943c3-22e8-4f34-bc42-0ce39fc0e8c2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:28.202515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 619/1250] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:16:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:28.208763 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3566a17-decf-474c-b7ad-4f0c8e08e2ee None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:28.297310 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:28.329982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3566a17-decf-474c-b7ad-4f0c8e08e2ee None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:28.367372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c103d12-f430-4deb-a926-e6a792f77ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:28.368033 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c103d12-f430-4deb-a926-e6a792f77ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:28.368475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c103d12-f430-4deb-a926-e6a792f77ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:28.368757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c103d12-f430-4deb-a926-e6a792f77ce7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:28.383366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c103d12-f430-4deb-a926-e6a792f77ce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:28.385133 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c103d12-f430-4deb-a926-e6a792f77ce7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:28.406556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-2c103d12-f430-4deb-a926-e6a792f77ce7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:28.406556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:28.406556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:28.406556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:28.406556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:28.406556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:28.406556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:28.406556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:28.406556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:28.406556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:28.406556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:28.406556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:28.406556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:16:28.406556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:16:28.406556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:16:28.406556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:16:28.408027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:28.408027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:28.408027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:28.408027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:28.408027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:28.408027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:28.408027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:28.408027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:28.408027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:28.408027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:28.408027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:28.408027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:28.408027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:28.408027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:28.408027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:28.408027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:28.408027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:16:28.409278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:16:28.409278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:28.409278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:16:28.414287 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 632/1251] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:28.414443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7943c3-22e8-4f34-bc42-0ce39fc0e8c2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:28.526487 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7943c3-22e8-4f34-bc42-0ce39fc0e8c2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:28.561962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cbc88-965a-4f48-9f1a-2bba8a9a0aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:28.562548 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cbc88-965a-4f48-9f1a-2bba8a9a0aba None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:28.562828 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cbc88-965a-4f48-9f1a-2bba8a9a0aba None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:28.563100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cbc88-965a-4f48-9f1a-2bba8a9a0aba None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:28.564234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057cbc88-965a-4f48-9f1a-2bba8a9a0aba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:28.564543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057cbc88-965a-4f48-9f1a-2bba8a9a0aba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:28.565907 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057cbc88-965a-4f48-9f1a-2bba8a9a0aba None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:28.580447 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 620/1252] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:28.586473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c103d12-f430-4deb-a926-e6a792f77ce7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:28.709662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c103d12-f430-4deb-a926-e6a792f77ce7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:28.741811 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:28.752496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62634344-3b63-4bac-98cf-0a4aaabbf45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:28.753137 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62634344-3b63-4bac-98cf-0a4aaabbf45d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:28.753535 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62634344-3b63-4bac-98cf-0a4aaabbf45d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:28.753812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62634344-3b63-4bac-98cf-0a4aaabbf45d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:28.764106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62634344-3b63-4bac-98cf-0a4aaabbf45d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:28.766342 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62634344-3b63-4bac-98cf-0a4aaabbf45d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:28.779101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-62634344-3b63-4bac-98cf-0a4aaabbf45d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:16:28.779101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:28.779101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:28.779101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:28.779101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:28.779101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:28.779101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:28.779101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:28.779101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:28.779101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:28.779101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:28.779101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:28.779101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:16:28.779101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:16:28.779101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:16:28.779101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:16:28.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:28.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:28.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:16:28.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:16:28.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:16:28.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:16:28.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:28.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:28.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:28.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:28.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:28.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:28.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:28.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:28.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:28.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:28.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:28.780126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:28.781337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:28.781337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:28.781337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:16:28.781337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:16:28.781337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:16:28.781337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:16:28.781337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:16:28.781337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:16:28.781337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:16:28.781337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:16:28.781337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 633/1253] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:28.787440 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cbc88-965a-4f48-9f1a-2bba8a9a0aba None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:28.852232 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:28.902237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cbc88-965a-4f48-9f1a-2bba8a9a0aba None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:28.941594 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51357b9b-a398-4b72-881a-0c51632effaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:28.942509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51357b9b-a398-4b72-881a-0c51632effaa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:28.942778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51357b9b-a398-4b72-881a-0c51632effaa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:28.943042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51357b9b-a398-4b72-881a-0c51632effaa None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:28.943755 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51357b9b-a398-4b72-881a-0c51632effaa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:28.944050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51357b9b-a398-4b72-881a-0c51632effaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:28.945506 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51357b9b-a398-4b72-881a-0c51632effaa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:28.962578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 621/1254] 10.222.0.39 () {56 vars in 1143 bytes} [Mon Aug 8 02:16:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:28.968684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62634344-3b63-4bac-98cf-0a4aaabbf45d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:29.073792 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62634344-3b63-4bac-98cf-0a4aaabbf45d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:29.122852 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8f4c2-0515-4688-a840-b6563bc28688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:29.123670 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8f4c2-0515-4688-a840-b6563bc28688 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:29.124179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8f4c2-0515-4688-a840-b6563bc28688 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:29.124679 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8f4c2-0515-4688-a840-b6563bc28688 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:29.191768 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:29.204961 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8f4c2-0515-4688-a840-b6563bc28688 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:29.206442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8f4c2-0515-4688-a840-b6563bc28688 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:29.317631 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 634/1255] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:16:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:16:29.323747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51357b9b-a398-4b72-881a-0c51632effaa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:29.434235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51357b9b-a398-4b72-881a-0c51632effaa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:29.449600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:29.474051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689a3f0-9182-4e5e-8d69-d56101529286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:29.474732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689a3f0-9182-4e5e-8d69-d56101529286 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:29.475004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689a3f0-9182-4e5e-8d69-d56101529286 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:29.475270 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689a3f0-9182-4e5e-8d69-d56101529286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:29.486115 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689a3f0-9182-4e5e-8d69-d56101529286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:29.488172 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689a3f0-9182-4e5e-8d69-d56101529286 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:29.502809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3689a3f0-9182-4e5e-8d69-d56101529286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:29.502809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:29.502809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:29.502809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:29.502809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:29.502809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:29.502809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:29.502809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:29.502809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:29.502809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:29.502809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:29.502809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:29.502809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:16:29.502809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:16:29.502809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:16:29.502809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:16:29.503834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:29.503834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:29.503834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:29.503834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:29.503834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:29.503834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:29.503834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:29.503834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:29.503834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:29.503834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:29.503834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:29.503834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:29.503834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:29.503834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:29.503834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:29.503834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:29.503834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:16:29.504751 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:16:29.504751 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:29.504751 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:29.509015 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 622/1256] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:29.510025 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8f4c2-0515-4688-a840-b6563bc28688 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:29.601664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:29.614762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8f4c2-0515-4688-a840-b6563bc28688 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:29.650694 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ada18-4882-4efc-855d-830e4c8bfd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:29.651303 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ada18-4882-4efc-855d-830e4c8bfd22 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:29.651580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ada18-4882-4efc-855d-830e4c8bfd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:29.651851 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ada18-4882-4efc-855d-830e4c8bfd22 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:29.652491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ada18-4882-4efc-855d-830e4c8bfd22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:29.652796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ada18-4882-4efc-855d-830e4c8bfd22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:29.654158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ada18-4882-4efc-855d-830e4c8bfd22 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:29.670288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 635/1257] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:29.672637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689a3f0-9182-4e5e-8d69-d56101529286 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:29.782372 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689a3f0-9182-4e5e-8d69-d56101529286 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:29.817842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b81d6b-3110-491e-a3e4-662121fa1a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:29.818620 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b81d6b-3110-491e-a3e4-662121fa1a03 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:29.818970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b81d6b-3110-491e-a3e4-662121fa1a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:29.819289 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b81d6b-3110-491e-a3e4-662121fa1a03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:29.827484 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b81d6b-3110-491e-a3e4-662121fa1a03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:29.829199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b81d6b-3110-491e-a3e4-662121fa1a03 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:29.840894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b9b81d6b-3110-491e-a3e4-662121fa1a03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:16:29.840894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:29.840894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:29.840894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:29.840894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:29.840894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:29.840894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:29.840894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:29.840894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:29.840894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:29.840894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:29.840894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:29.840894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:16:29.840894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:16:29.840894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:16:29.842055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:16:29.842055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:29.842055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:29.842055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:29.842055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:29.842055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:29.842055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:29.842055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:29.842055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:29.842055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:29.842055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:29.842055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:29.842055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:29.842055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:29.842055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:29.842055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:29.842055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:29.842983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:16:29.842983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:16:29.842983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:16:29.842983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:16:29.842983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:16:29.842983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:16:29.842983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:16:29.842983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:29.842983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 623/1258] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:16:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:29.847990 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ada18-4882-4efc-855d-830e4c8bfd22 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:29.954992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ada18-4882-4efc-855d-830e4c8bfd22 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:29.988283 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8e676-243b-4e1b-ae40-e3323949514f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:29.988906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8e676-243b-4e1b-ae40-e3323949514f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:29.989173 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8e676-243b-4e1b-ae40-e3323949514f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:29.989440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8e676-243b-4e1b-ae40-e3323949514f None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:29.990118 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8e676-243b-4e1b-ae40-e3323949514f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:29.990451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8e676-243b-4e1b-ae40-e3323949514f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:29.991858 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8e676-243b-4e1b-ae40-e3323949514f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:30.004766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 636/1259] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:16:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:30.011054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b81d6b-3110-491e-a3e4-662121fa1a03 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:30.040626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:30.129348 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b81d6b-3110-491e-a3e4-662121fa1a03 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:30.172934 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccc4b2-e7e3-4d8d-921b-aaacc6be5a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:30.174107 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccc4b2-e7e3-4d8d-921b-aaacc6be5a5d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:30.174497 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccc4b2-e7e3-4d8d-921b-aaacc6be5a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:30.174870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccc4b2-e7e3-4d8d-921b-aaacc6be5a5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:30.175799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaccc4b2-e7e3-4d8d-921b-aaacc6be5a5d None admin] RBAC: Adding query filter params (role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=f62745b92a864480a9438de581e1d05e) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:30.176190 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaccc4b2-e7e3-4d8d-921b-aaacc6be5a5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:30.178021 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaccc4b2-e7e3-4d8d-921b-aaacc6be5a5d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:30.185464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 624/1260] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:16:30 2022] GET /identity/v3/role_assignments?user.id=f62745b92a864480a9438de581e1d05e&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:32.184438 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c14d6-3a42-4195-a98d-9425a2f5d825 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:32.185234 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c14d6-3a42-4195-a98d-9425a2f5d825 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:32.185234 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c14d6-3a42-4195-a98d-9425a2f5d825 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:32.186597 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 637/1261] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:32.191653 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9a083-fa16-4700-beb5-170c501f8655 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:32.192031 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9a083-fa16-4700-beb5-170c501f8655 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:32.192193 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9a083-fa16-4700-beb5-170c501f8655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:32.209679 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-13d9a083-fa16-4700-beb5-170c501f8655 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:32.210865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-13d9a083-fa16-4700-beb5-170c501f8655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:32.275443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 625/1262] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:32.297744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c14d6-3a42-4195-a98d-9425a2f5d825 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:32.309002 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:32.396778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c14d6-3a42-4195-a98d-9425a2f5d825 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:32.429800 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bff6c7-c9c4-4058-ad3a-4da88e97044f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:32.430437 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bff6c7-c9c4-4058-ad3a-4da88e97044f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:32.430710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bff6c7-c9c4-4058-ad3a-4da88e97044f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:32.430974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bff6c7-c9c4-4058-ad3a-4da88e97044f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:32.440980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bff6c7-c9c4-4058-ad3a-4da88e97044f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:16:32.445636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-f3bff6c7-c9c4-4058-ad3a-4da88e97044f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:16:32.445636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:32.445636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:32.445636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:32.445636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:32.445636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:32.445636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:32.445636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:32.445636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:32.445636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:32.445636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:32.445636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:32.445636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:16:32.445636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:16:32.445636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:16:32.445636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:16:32.446770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:16:32.446770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:16:32.446770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:16:32.446770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:16:32.446770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:32.446770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:32.446770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:32.446770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:32.446770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:32.446770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:32.446770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:32.446770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:32.446770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:32.446770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:32.446770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:32.446770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:32.447788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:32.447788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:32.447788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:32.447788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:32.447788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:16:32.447788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:16:32.447788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:16:32.447788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:16:32.447788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:16:32.447788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:16:32.447788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:16:32.447788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:16:32.447788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 638/1263] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:16:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:32.452806 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9a083-fa16-4700-beb5-170c501f8655 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:32.471457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:32.549195 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9a083-fa16-4700-beb5-170c501f8655 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:32.581937 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e51d2-0e50-4072-b477-ff40083dde09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:32.582548 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e51d2-0e50-4072-b477-ff40083dde09 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:32.582824 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e51d2-0e50-4072-b477-ff40083dde09 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:32.583087 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e51d2-0e50-4072-b477-ff40083dde09 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:32.583727 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e51d2-0e50-4072-b477-ff40083dde09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:32.584024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e51d2-0e50-4072-b477-ff40083dde09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:32.585366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e51d2-0e50-4072-b477-ff40083dde09 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:32.592959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 626/1264] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:16:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:32.599106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bff6c7-c9c4-4058-ad3a-4da88e97044f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:32.663534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:32.695851 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bff6c7-c9c4-4058-ad3a-4da88e97044f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:32.728502 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534eeda-3aa8-4b8c-b0d8-9f6081b576d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:32.729105 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534eeda-3aa8-4b8c-b0d8-9f6081b576d0 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:32.729380 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534eeda-3aa8-4b8c-b0d8-9f6081b576d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:32.729647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534eeda-3aa8-4b8c-b0d8-9f6081b576d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:32.739800 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534eeda-3aa8-4b8c-b0d8-9f6081b576d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:32.741249 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534eeda-3aa8-4b8c-b0d8-9f6081b576d0 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:32.755168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-8534eeda-3aa8-4b8c-b0d8-9f6081b576d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:32.755168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:32.755168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:32.755168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:32.755168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:32.755168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:32.755168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:32.755168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:32.755168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:32.755168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:32.755168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:32.755168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:32.755168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:16:32.755168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:16:32.755168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:16:32.755168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:16:32.756102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:32.756102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:32.756102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:32.756102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:32.756102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:32.756102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:32.756102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:32.756102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:32.756102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:32.756102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:32.756102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:32.756102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:32.756102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:32.756102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:32.756102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:32.756102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:32.756102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:16:32.757199 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:16:32.757199 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:32.757199 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:16:32.757199 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 639/1265] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:32.761803 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e51d2-0e50-4072-b477-ff40083dde09 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:32.839617 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:32.858153 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e51d2-0e50-4072-b477-ff40083dde09 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:32.890103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a1fcd-63b7-4afd-a682-f92abb141b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:32.890726 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a1fcd-63b7-4afd-a682-f92abb141b74 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:32.891001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a1fcd-63b7-4afd-a682-f92abb141b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:32.891265 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a1fcd-63b7-4afd-a682-f92abb141b74 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:32.892461 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a1fcd-63b7-4afd-a682-f92abb141b74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:32.892759 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a1fcd-63b7-4afd-a682-f92abb141b74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:32.894097 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a1fcd-63b7-4afd-a682-f92abb141b74 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:32.905196 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 627/1266] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:32.910603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534eeda-3aa8-4b8c-b0d8-9f6081b576d0 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:33.006824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534eeda-3aa8-4b8c-b0d8-9f6081b576d0 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:33.025631 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:33.039504 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3bb659-5fe9-4209-8252-2f9dbd4ebc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:33.040104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bb659-5fe9-4209-8252-2f9dbd4ebc89 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:33.040375 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bb659-5fe9-4209-8252-2f9dbd4ebc89 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:33.040638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bb659-5fe9-4209-8252-2f9dbd4ebc89 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:33.047700 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3bb659-5fe9-4209-8252-2f9dbd4ebc89 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:33.049135 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3bb659-5fe9-4209-8252-2f9dbd4ebc89 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:33.059936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-5f3bb659-5fe9-4209-8252-2f9dbd4ebc89 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:16:33.059936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:33.059936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:33.059936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:33.059936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:33.059936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:33.059936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:33.059936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:33.059936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:33.059936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:33.059936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:33.059936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:33.059936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:16:33.059936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:16:33.059936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:16:33.059936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:16:33.061041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:33.061041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:33.061041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:16:33.061041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:16:33.061041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:16:33.061041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:16:33.061041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:33.061041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:33.061041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:33.061041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:33.061041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:33.061041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:33.061041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:33.061041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:33.061041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:33.061041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:33.061041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:33.061041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:33.062325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:33.062325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:33.062325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:16:33.062325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:16:33.062325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:16:33.062325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:16:33.062325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:16:33.062325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:16:33.062325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:16:33.062325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:16:33.062325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 640/1267] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:33.067093 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a1fcd-63b7-4afd-a682-f92abb141b74 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:33.173658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a1fcd-63b7-4afd-a682-f92abb141b74 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:33.207508 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcd704-23d5-4cb7-962a-7f8fcecdd250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:33.208089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcd704-23d5-4cb7-962a-7f8fcecdd250 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:33.208350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcd704-23d5-4cb7-962a-7f8fcecdd250 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:33.208615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcd704-23d5-4cb7-962a-7f8fcecdd250 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:33.209320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fcd704-23d5-4cb7-962a-7f8fcecdd250 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:33.209618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fcd704-23d5-4cb7-962a-7f8fcecdd250 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:33.211160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fcd704-23d5-4cb7-962a-7f8fcecdd250 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:33.219332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:33.227813 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 628/1268] 10.222.0.39 () {56 vars in 1143 bytes} [Mon Aug 8 02:16:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:33.233525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3bb659-5fe9-4209-8252-2f9dbd4ebc89 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:33.329966 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3bb659-5fe9-4209-8252-2f9dbd4ebc89 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:33.361644 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac2b082-bbdd-4f4f-9184-70687aebae36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:33.362232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac2b082-bbdd-4f4f-9184-70687aebae36 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:33.362537 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac2b082-bbdd-4f4f-9184-70687aebae36 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:33.362804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac2b082-bbdd-4f4f-9184-70687aebae36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:33.372754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac2b082-bbdd-4f4f-9184-70687aebae36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:33.374162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac2b082-bbdd-4f4f-9184-70687aebae36 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:33.388042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-eac2b082-bbdd-4f4f-9184-70687aebae36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:33.388042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:33.388042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:33.388042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:33.388042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:33.388042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:33.388042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:33.388042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:33.388042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:33.388042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:33.388042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:33.388042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:33.388042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:16:33.388042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:16:33.388042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:16:33.388042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:16:33.392920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:33.392920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:33.392920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:33.392920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:33.392920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:33.392920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:33.392920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:33.392920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:33.392920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:33.392920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:33.392920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:33.392920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:33.392920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:33.392920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:33.392920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:33.392920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:33.392920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:16:33.393985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:16:33.393985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:33.393985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:16:33.393985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 641/1269] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:33.394765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcd704-23d5-4cb7-962a-7f8fcecdd250 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:33.490834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcd704-23d5-4cb7-962a-7f8fcecdd250 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:33.522568 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3cf157-71ae-4717-9418-01a4e8074ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:33.523148 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cf157-71ae-4717-9418-01a4e8074ede None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:33.523422 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cf157-71ae-4717-9418-01a4e8074ede None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:33.523688 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cf157-71ae-4717-9418-01a4e8074ede None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:33.524313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3cf157-71ae-4717-9418-01a4e8074ede None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:33.524609 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3cf157-71ae-4717-9418-01a4e8074ede None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:33.525952 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3cf157-71ae-4717-9418-01a4e8074ede None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:33.537057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 629/1270] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:33.542615 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac2b082-bbdd-4f4f-9184-70687aebae36 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:33.573748 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:33.640333 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac2b082-bbdd-4f4f-9184-70687aebae36 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:33.673727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e05612-ebcb-4a89-9f71-8e18f44df8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:33.674366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e05612-ebcb-4a89-9f71-8e18f44df8a7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:33.674643 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e05612-ebcb-4a89-9f71-8e18f44df8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:33.674910 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e05612-ebcb-4a89-9f71-8e18f44df8a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:33.681290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e05612-ebcb-4a89-9f71-8e18f44df8a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:33.682762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e05612-ebcb-4a89-9f71-8e18f44df8a7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:33.694262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-55e05612-ebcb-4a89-9f71-8e18f44df8a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:16:33.694262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:33.694262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:33.694262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:33.694262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:33.694262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:33.694262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:33.694262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:33.694262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:33.694262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:33.694262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:33.694262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:33.694262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:16:33.694262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:16:33.694262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:16:33.695280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:16:33.695280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:33.695280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:33.695280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:33.695280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:33.695280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:33.695280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:33.695280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:33.695280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:33.695280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:33.695280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:33.695280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:33.695280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:33.695280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:33.695280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:33.695280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:33.695280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:33.696204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:16:33.696204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:16:33.696204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:16:33.696204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:16:33.696204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:16:33.696204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:16:33.696204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:16:33.696204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:16:33.696204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 642/1271] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:16:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:33.701227 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3cf157-71ae-4717-9418-01a4e8074ede None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:33.758574 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:33.799174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3cf157-71ae-4717-9418-01a4e8074ede None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:33.830628 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e78d41-1648-4a6d-a94c-40ae5dbc998b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:33.831209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e78d41-1648-4a6d-a94c-40ae5dbc998b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:33.831480 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e78d41-1648-4a6d-a94c-40ae5dbc998b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:33.831743 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e78d41-1648-4a6d-a94c-40ae5dbc998b None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:33.832411 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e78d41-1648-4a6d-a94c-40ae5dbc998b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:33.832713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e78d41-1648-4a6d-a94c-40ae5dbc998b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:33.834119 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e78d41-1648-4a6d-a94c-40ae5dbc998b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:33.845866 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 630/1272] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:16:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:33.851634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e05612-ebcb-4a89-9f71-8e18f44df8a7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:33.939190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:33.947738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e05612-ebcb-4a89-9f71-8e18f44df8a7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:33.980221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81268341-1442-4f1c-a957-e14489f5af8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:33.980846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81268341-1442-4f1c-a957-e14489f5af8e None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:33.981123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81268341-1442-4f1c-a957-e14489f5af8e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:33.981393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81268341-1442-4f1c-a957-e14489f5af8e None admin] PATH_INFO: `/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/f62745b92a864480a9438de581e1d05e/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:33.983283 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81268341-1442-4f1c-a957-e14489f5af8e None admin] RBAC: Authorizing `identity:create_grant(project_id=7d474b61f1fe4f17aacb7a3665315c79, user_id=f62745b92a864480a9438de581e1d05e, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:33.984689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81268341-1442-4f1c-a957-e14489f5af8e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:33.996372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 643/1273] 10.222.0.39 () {58 vars in 1439 bytes} [Mon Aug 8 02:16:33 2022] PUT /identity/v3/projects/7d474b61f1fe4f17aacb7a3665315c79/users/f62745b92a864480a9438de581e1d05e/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:16:35.927392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0331551-f3d6-4932-bc72-06ab956d3cea None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:35.927939 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0331551-f3d6-4932-bc72-06ab956d3cea None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:35.927939 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0331551-f3d6-4932-bc72-06ab956d3cea None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:35.929248 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 631/1274] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:35.934324 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a471d41-aba3-4b0f-9451-20f42ae40045 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:35.934990 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a471d41-aba3-4b0f-9451-20f42ae40045 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:35.935120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a471d41-aba3-4b0f-9451-20f42ae40045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:35.953356 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-9a471d41-aba3-4b0f-9451-20f42ae40045 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:35.955168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9a471d41-aba3-4b0f-9451-20f42ae40045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:36.019127 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 644/1275] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:36.041385 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0331551-f3d6-4932-bc72-06ab956d3cea None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:36.147322 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0331551-f3d6-4932-bc72-06ab956d3cea None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:36.165490 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:36.182770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b54dc-b82f-40c3-8589-f3e49fc3a7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:36.183330 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b54dc-b82f-40c3-8589-f3e49fc3a7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:36.183608 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b54dc-b82f-40c3-8589-f3e49fc3a7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:36.183878 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b54dc-b82f-40c3-8589-f3e49fc3a7b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:36.194457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b54dc-b82f-40c3-8589-f3e49fc3a7b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:16:36.199084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-cf4b54dc-b82f-40c3-8589-f3e49fc3a7b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:16:36.199084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:36.199084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:36.199084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:36.199084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:36.199084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:36.199084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:36.199084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:36.199084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:36.199084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:36.199084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:36.199084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:36.199084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:16:36.199084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:16:36.199084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:16:36.199084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:16:36.200157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:16:36.200157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:16:36.200157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:16:36.200157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:16:36.200157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:36.200157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:36.200157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:36.200157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:36.200157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:36.200157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:36.200157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:36.200157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:36.200157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:36.200157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:36.200157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:36.200157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:36.201181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:36.201181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:36.201181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:36.201181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:36.201181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:16:36.201181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:16:36.201181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:16:36.201181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:16:36.201181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:16:36.201181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:16:36.201181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:16:36.201181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:36.201181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 632/1276] 10.222.0.39 () {56 vars in 1096 bytes} [Mon Aug 8 02:16:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:36.206383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a471d41-aba3-4b0f-9451-20f42ae40045 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:36.282515 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:36.301309 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a471d41-aba3-4b0f-9451-20f42ae40045 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:36.334044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0638e0-9735-4ae7-b052-8f5f2bebb3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:36.334663 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0638e0-9735-4ae7-b052-8f5f2bebb3ac None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:36.334941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0638e0-9735-4ae7-b052-8f5f2bebb3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:36.335211 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0638e0-9735-4ae7-b052-8f5f2bebb3ac None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:36.335861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0638e0-9735-4ae7-b052-8f5f2bebb3ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:36.336162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0638e0-9735-4ae7-b052-8f5f2bebb3ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:36.337511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0638e0-9735-4ae7-b052-8f5f2bebb3ac None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:36.345193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 645/1277] 10.222.0.39 () {56 vars in 1099 bytes} [Mon Aug 8 02:16:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:36.350793 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b54dc-b82f-40c3-8589-f3e49fc3a7b6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:36.453989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b54dc-b82f-40c3-8589-f3e49fc3a7b6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:36.487264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcdca6-460f-4581-babc-27a4dcd4f0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:36.487879 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcdca6-460f-4581-babc-27a4dcd4f0ab None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:36.488164 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcdca6-460f-4581-babc-27a4dcd4f0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:36.488433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcdca6-460f-4581-babc-27a4dcd4f0ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:36.498136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adcdca6-460f-4581-babc-27a4dcd4f0ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:36.499616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adcdca6-460f-4581-babc-27a4dcd4f0ab None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:36.514168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-2adcdca6-460f-4581-babc-27a4dcd4f0ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:36.514168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:36.514168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:36.514168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:36.514168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:36.514168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:36.514168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:36.514168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:36.514168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:36.514168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:36.514168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:36.514168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:36.514168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:16:36.514168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:16:36.514168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:16:36.514168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:16:36.515305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:36.515305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:36.515305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:36.515305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:36.515305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:36.515305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:36.515305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:36.515305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:36.515305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:36.515305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:36.515305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:36.515305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:36.515305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:36.515305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:36.515305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:36.515305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:36.515305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:16:36.516349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:16:36.516349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:36.516349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:36.516349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 633/1278] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:36.520922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0638e0-9735-4ae7-b052-8f5f2bebb3ac None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:36.617837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0638e0-9735-4ae7-b052-8f5f2bebb3ac None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:36.649215 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101265e-6d3d-4093-960f-5667c59196f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:36.649799 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101265e-6d3d-4093-960f-5667c59196f3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:36.650075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101265e-6d3d-4093-960f-5667c59196f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:36.650378 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101265e-6d3d-4093-960f-5667c59196f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:36.651011 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101265e-6d3d-4093-960f-5667c59196f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:36.651314 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101265e-6d3d-4093-960f-5667c59196f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:36.652647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101265e-6d3d-4093-960f-5667c59196f3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:36.664326 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 646/1279] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:36.669654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcdca6-460f-4581-babc-27a4dcd4f0ab None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:36.699824 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:36.767356 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcdca6-460f-4581-babc-27a4dcd4f0ab None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:36.799849 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274653a-7d21-4807-90c3-44d97f432fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:36.800454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274653a-7d21-4807-90c3-44d97f432fdb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:36.800726 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274653a-7d21-4807-90c3-44d97f432fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:36.800991 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274653a-7d21-4807-90c3-44d97f432fdb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:36.807994 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274653a-7d21-4807-90c3-44d97f432fdb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:36.809433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274653a-7d21-4807-90c3-44d97f432fdb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:36.820215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3274653a-7d21-4807-90c3-44d97f432fdb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:16:36.820215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:36.820215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:36.820215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:36.820215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:36.820215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:36.820215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:36.820215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:36.820215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:36.820215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:36.820215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:36.820215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:36.820215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:16:36.820215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:16:36.820215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:16:36.820215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:16:36.821315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:36.821315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:36.821315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:16:36.821315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:16:36.821315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:16:36.821315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:16:36.821315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:36.821315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:36.821315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:36.821315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:36.821315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:36.821315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:36.821315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:36.821315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:36.821315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:36.821315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:36.821315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:36.821315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:36.822680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:36.822680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:36.822680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:16:36.822680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:16:36.822680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:16:36.822680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:16:36.822680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:16:36.822680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:16:36.822680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:16:36.822680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:36.822680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 634/1280] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:36.827117 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101265e-6d3d-4093-960f-5667c59196f3 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:36.830882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:36.922166 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101265e-6d3d-4093-960f-5667c59196f3 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:36.954563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64836160-afd5-4b36-9350-82c8d505ed7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:36.955127 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64836160-afd5-4b36-9350-82c8d505ed7d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:36.955363 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64836160-afd5-4b36-9350-82c8d505ed7d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:36.956533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64836160-afd5-4b36-9350-82c8d505ed7d None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:36.956533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64836160-afd5-4b36-9350-82c8d505ed7d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:36.956719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64836160-afd5-4b36-9350-82c8d505ed7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:36.958012 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64836160-afd5-4b36-9350-82c8d505ed7d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:36.975474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 647/1281] 10.222.0.39 () {56 vars in 1143 bytes} [Mon Aug 8 02:16:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:36.981186 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274653a-7d21-4807-90c3-44d97f432fdb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:37.066394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:37.076152 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274653a-7d21-4807-90c3-44d97f432fdb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:37.113207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e3d86-882e-4061-9d4a-ba556d580d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:37.113835 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e3d86-882e-4061-9d4a-ba556d580d36 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:37.114120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e3d86-882e-4061-9d4a-ba556d580d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:37.114572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e3d86-882e-4061-9d4a-ba556d580d36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:37.179568 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e3d86-882e-4061-9d4a-ba556d580d36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:37.181012 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e3d86-882e-4061-9d4a-ba556d580d36 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:37.265909 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:37.276632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 635/1282] 10.222.0.39 () {58 vars in 1282 bytes} [Mon Aug 8 02:16:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:16:37.282508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64836160-afd5-4b36-9350-82c8d505ed7d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:37.341439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:37.378076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64836160-afd5-4b36-9350-82c8d505ed7d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:37.410348 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ca67f-c928-4a5e-85b6-47397bd8b680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:37.410953 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ca67f-c928-4a5e-85b6-47397bd8b680 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:37.411229 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ca67f-c928-4a5e-85b6-47397bd8b680 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:37.411496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ca67f-c928-4a5e-85b6-47397bd8b680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:37.421181 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ca67f-c928-4a5e-85b6-47397bd8b680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:37.422626 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ca67f-c928-4a5e-85b6-47397bd8b680 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:37.436798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-032ca67f-c928-4a5e-85b6-47397bd8b680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:37.436798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:37.436798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:37.436798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:37.436798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:37.436798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:37.436798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:37.436798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:37.436798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:37.436798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:37.436798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:37.436798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:37.436798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:16:37.436798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:16:37.436798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:16:37.436798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:16:37.437727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:37.437727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:37.437727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:37.437727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:37.437727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:37.437727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:37.437727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:37.437727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:37.437727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:37.437727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:37.437727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:37.437727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:37.437727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:37.437727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:37.437727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:37.437727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:37.437727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:16:37.439207 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:16:37.439207 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:16:37.439207 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:16:37.439207 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 648/1283] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:37.444002 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e3d86-882e-4061-9d4a-ba556d580d36 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:37.538944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e3d86-882e-4061-9d4a-ba556d580d36 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:37.571573 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c80b2-141e-4154-9f6c-c33f66a986c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:37.572158 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c80b2-141e-4154-9f6c-c33f66a986c8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:37.572438 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c80b2-141e-4154-9f6c-c33f66a986c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:37.572709 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c80b2-141e-4154-9f6c-c33f66a986c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:37.573343 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c80b2-141e-4154-9f6c-c33f66a986c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:37.573644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c80b2-141e-4154-9f6c-c33f66a986c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:37.575049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c80b2-141e-4154-9f6c-c33f66a986c8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:37.586375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 636/1284] 10.222.0.39 () {56 vars in 1109 bytes} [Mon Aug 8 02:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:37.591811 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ca67f-c928-4a5e-85b6-47397bd8b680 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:37.689217 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ca67f-c928-4a5e-85b6-47397bd8b680 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:37.702404 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:37.721764 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f498c7-d963-4b2e-98ce-c429a7b0c44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:37.722403 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f498c7-d963-4b2e-98ce-c429a7b0c44c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:37.722681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f498c7-d963-4b2e-98ce-c429a7b0c44c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:37.722952 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f498c7-d963-4b2e-98ce-c429a7b0c44c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:37.729336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f498c7-d963-4b2e-98ce-c429a7b0c44c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:37.730796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f498c7-d963-4b2e-98ce-c429a7b0c44c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:37.742917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-79f498c7-d963-4b2e-98ce-c429a7b0c44c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:16:37.742917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:37.742917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:37.742917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:37.742917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:37.742917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:37.742917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:37.742917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:37.742917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:37.742917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:37.742917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:37.742917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:37.742917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:16:37.742917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:16:37.742917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:16:37.744019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:16:37.744019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:37.744019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:37.744019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:37.744019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:37.744019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:37.744019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:37.744019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:37.744019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:37.744019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:37.744019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:37.744019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:37.744019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:37.744019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:37.744019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:37.744019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:37.744019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:37.745147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:16:37.745147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:16:37.745147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:16:37.745147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:16:37.745147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:16:37.745147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:16:37.745147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:16:37.745147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 08 02:16:37.745147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 649/1285] 10.222.0.39 () {56 vars in 1111 bytes} [Mon Aug 8 02:16:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:37.749811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c80b2-141e-4154-9f6c-c33f66a986c8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:37.798802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:37.848026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c80b2-141e-4154-9f6c-c33f66a986c8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:37.881691 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5155ff0-e0d0-4aee-b348-e82deab43a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:37.882312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5155ff0-e0d0-4aee-b348-e82deab43a91 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:37.882595 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5155ff0-e0d0-4aee-b348-e82deab43a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:37.882865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5155ff0-e0d0-4aee-b348-e82deab43a91 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:37.883540 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5155ff0-e0d0-4aee-b348-e82deab43a91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:37.883844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5155ff0-e0d0-4aee-b348-e82deab43a91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:37.885277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5155ff0-e0d0-4aee-b348-e82deab43a91 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:37.896993 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 637/1286] 10.222.0.39 () {56 vars in 1148 bytes} [Mon Aug 8 02:16:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:37.903056 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f498c7-d963-4b2e-98ce-c429a7b0c44c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:38.000690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f498c7-d963-4b2e-98ce-c429a7b0c44c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:38.033546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f1c04-bac0-45f5-9cd3-2ec046189fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:38.034133 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f1c04-bac0-45f5-9cd3-2ec046189fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:38.034442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f1c04-bac0-45f5-9cd3-2ec046189fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:38.034717 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f1c04-bac0-45f5-9cd3-2ec046189fa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:38.035438 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f1c04-bac0-45f5-9cd3-2ec046189fa9 None admin] RBAC: Adding query filter params (role.id=ebae0c49752b42c7b7b72945e76d9fe3, scope.project.id=7d474b61f1fe4f17aacb7a3665315c79, user.id=f62745b92a864480a9438de581e1d05e) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:38.035746 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f1c04-bac0-45f5-9cd3-2ec046189fa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:38.037169 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f1c04-bac0-45f5-9cd3-2ec046189fa9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:38.042850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 650/1287] 10.222.0.39 () {56 vars in 1374 bytes} [Mon Aug 8 02:16:37 2022] GET /identity/v3/role_assignments?user.id=f62745b92a864480a9438de581e1d05e&scope.project.id=7d474b61f1fe4f17aacb7a3665315c79&role.id=ebae0c49752b42c7b7b72945e76d9fe3 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:40.034694 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d90efd-4502-4158-ab4c-28227569efe9 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:40.035250 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d90efd-4502-4158-ab4c-28227569efe9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:40.035250 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d90efd-4502-4158-ab4c-28227569efe9 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:40.036565 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 638/1288] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:40.041767 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe48455-c94b-4faf-970c-680415d2cace None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:40.042191 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe48455-c94b-4faf-970c-680415d2cace None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:40.042326 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe48455-c94b-4faf-970c-680415d2cace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:40.060578 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-afe48455-c94b-4faf-970c-680415d2cace None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:40.061740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-afe48455-c94b-4faf-970c-680415d2cace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:40.090972 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:40.133573 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 651/1289] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:16:40.155148 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d90efd-4502-4158-ab4c-28227569efe9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:40.254630 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d90efd-4502-4158-ab4c-28227569efe9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:40.264787 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:40.288187 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93828f-bd2a-4b9a-a850-d483841a9116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:40.288733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93828f-bd2a-4b9a-a850-d483841a9116 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:40.289010 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93828f-bd2a-4b9a-a850-d483841a9116 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:40.289276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93828f-bd2a-4b9a-a850-d483841a9116 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:40.289849 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93828f-bd2a-4b9a-a850-d483841a9116 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:40.291242 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93828f-bd2a-4b9a-a850-d483841a9116 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:40.300295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6d93828f-bd2a-4b9a-a850-d483841a9116 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:16:40.300295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:40.300295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:16:40.300295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:16:40.300295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:16:40.300295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:16:40.300295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:16:40.300295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:16:40.300295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:16:40.300295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:40.300295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:16:40.300295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:40.300295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:40.300295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:40.300295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:40.300295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:40.300295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:40.301504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:40.301504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:40.301504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:40.301504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:40.301504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:40.301504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:40.301504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:16:40.301504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:16:40.301504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:16:40.301504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:16:40.301504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:40.301504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:40.301504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:40.301504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:40.301504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:40.301504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:40.302601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:40.302601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:40.302601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:40.302601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:40.302601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:40.302601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:40.302601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:40.302601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:40.302601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:40.302601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:40.302601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:16:40.302601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:16:40.302601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:16:40.302601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:40.302601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 639/1290] 10.222.0.39 () {56 vars in 1117 bytes} [Mon Aug 8 02:16:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:40.307300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe48455-c94b-4faf-970c-680415d2cace None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:40.401541 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe48455-c94b-4faf-970c-680415d2cace None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:40.436324 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e2905-a0d6-4d83-9c60-202c85f30600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:40.436919 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e2905-a0d6-4d83-9c60-202c85f30600 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:40.437192 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e2905-a0d6-4d83-9c60-202c85f30600 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:40.437458 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e2905-a0d6-4d83-9c60-202c85f30600 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:40.438072 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e2905-a0d6-4d83-9c60-202c85f30600 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:40.438408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e2905-a0d6-4d83-9c60-202c85f30600 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:40.439794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e2905-a0d6-4d83-9c60-202c85f30600 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:40.444621 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 652/1291] 10.222.0.39 () {56 vars in 1116 bytes} [Mon Aug 8 02:16:40 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:16:40.450138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93828f-bd2a-4b9a-a850-d483841a9116 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:40.546603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93828f-bd2a-4b9a-a850-d483841a9116 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:40.578774 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a5dc5-ab0e-4a30-85ad-47a16835ba52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:40.579348 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a5dc5-ab0e-4a30-85ad-47a16835ba52 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:40.579621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a5dc5-ab0e-4a30-85ad-47a16835ba52 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:40.579886 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a5dc5-ab0e-4a30-85ad-47a16835ba52 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:40.580493 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a5dc5-ab0e-4a30-85ad-47a16835ba52 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:40.580792 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a5dc5-ab0e-4a30-85ad-47a16835ba52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:40.582140 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a5dc5-ab0e-4a30-85ad-47a16835ba52 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:40.586402 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 640/1292] 10.222.0.39 () {56 vars in 1116 bytes} [Mon Aug 8 02:16:40 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:42.554170 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc3a02-350d-4300-9456-9a2d8f6ff821 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:42.554693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc3a02-350d-4300-9456-9a2d8f6ff821 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:42.554869 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc3a02-350d-4300-9456-9a2d8f6ff821 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:42.556198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 653/1293] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:42.561435 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce24291-c6b0-45d5-923b-83d1dfc5907d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:42.561796 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce24291-c6b0-45d5-923b-83d1dfc5907d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:42.561982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce24291-c6b0-45d5-923b-83d1dfc5907d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:42.579976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5ce24291-c6b0-45d5-923b-83d1dfc5907d None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:42.581156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5ce24291-c6b0-45d5-923b-83d1dfc5907d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:42.610742 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:42.647458 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 641/1294] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:42.669906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc3a02-350d-4300-9456-9a2d8f6ff821 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:42.700776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:42.768997 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc3a02-350d-4300-9456-9a2d8f6ff821 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:42.801793 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f42b47-2557-44ee-8f4a-d2dee53bd925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:42.802406 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f42b47-2557-44ee-8f4a-d2dee53bd925 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:42.802684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f42b47-2557-44ee-8f4a-d2dee53bd925 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:42.802953 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f42b47-2557-44ee-8f4a-d2dee53bd925 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:42.803592 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f42b47-2557-44ee-8f4a-d2dee53bd925 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:42.804938 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f42b47-2557-44ee-8f4a-d2dee53bd925 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:42.819578 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 654/1295] 10.222.0.39 () {60 vars in 1141 bytes} [Mon Aug 8 02:16:42 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:16:44.818275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a466ab-b82e-4e83-89bb-56e8e8ff9695 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:44.818874 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a466ab-b82e-4e83-89bb-56e8e8ff9695 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:44.818874 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a466ab-b82e-4e83-89bb-56e8e8ff9695 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:44.820203 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 642/1296] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:44.825180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24330895-28a9-4bc9-bb18-cccd0b2424c0 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:44.825606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24330895-28a9-4bc9-bb18-cccd0b2424c0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:44.825742 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24330895-28a9-4bc9-bb18-cccd0b2424c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:44.843750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-24330895-28a9-4bc9-bb18-cccd0b2424c0 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:44.844955 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-24330895-28a9-4bc9-bb18-cccd0b2424c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:44.913513 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 655/1297] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:44 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:44.936417 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a466ab-b82e-4e83-89bb-56e8e8ff9695 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:45.037290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a466ab-b82e-4e83-89bb-56e8e8ff9695 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:45.069908 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e1581-9b83-40d6-b3e9-64b4cf3eb9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:45.070471 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e1581-9b83-40d6-b3e9-64b4cf3eb9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:45.070749 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e1581-9b83-40d6-b3e9-64b4cf3eb9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:45.071017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e1581-9b83-40d6-b3e9-64b4cf3eb9d0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:45.071587 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e1581-9b83-40d6-b3e9-64b4cf3eb9d0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:45.072937 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e1581-9b83-40d6-b3e9-64b4cf3eb9d0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:45.081848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-852e1581-9b83-40d6-b3e9-64b4cf3eb9d0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:16:45.081848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:45.081848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:16:45.081848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:16:45.081848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:16:45.081848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:16:45.081848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:16:45.081848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:16:45.081848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:16:45.081848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:45.081848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:16:45.081848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:45.081848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:45.081848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:45.081848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:45.081848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:45.081848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:45.083035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:45.083035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:45.083035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:45.083035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:45.083035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:45.083035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:45.083035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:16:45.083035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:16:45.083035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:16:45.083035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:16:45.083035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:45.083035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:45.083035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:45.083035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:45.083035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:45.083035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:45.083884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:45.083884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:45.083884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:45.083884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:45.083884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:45.083884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:45.083884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:45.083884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:45.083884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:45.083884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:45.083884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:16:45.083884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:16:45.083884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:16:45.083884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:45.083884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 643/1298] 10.222.0.39 () {56 vars in 1117 bytes} [Mon Aug 8 02:16:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:45.088734 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24330895-28a9-4bc9-bb18-cccd0b2424c0 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:45.108887 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:45.194295 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24330895-28a9-4bc9-bb18-cccd0b2424c0 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:45.226371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df21bf7-1768-40c5-ae4f-b691d293ecf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:45.226969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df21bf7-1768-40c5-ae4f-b691d293ecf1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:45.227244 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df21bf7-1768-40c5-ae4f-b691d293ecf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:45.227512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df21bf7-1768-40c5-ae4f-b691d293ecf1 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:45.228134 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df21bf7-1768-40c5-ae4f-b691d293ecf1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:45.228435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df21bf7-1768-40c5-ae4f-b691d293ecf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:45.229784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df21bf7-1768-40c5-ae4f-b691d293ecf1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:45.234741 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 656/1299] 10.222.0.39 () {56 vars in 1116 bytes} [Mon Aug 8 02:16:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:45.241361 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e1581-9b83-40d6-b3e9-64b4cf3eb9d0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:45.272484 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:45.340115 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e1581-9b83-40d6-b3e9-64b4cf3eb9d0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:45.372057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe49e3-a2a2-47a3-b203-21c69803887f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:45.372639 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe49e3-a2a2-47a3-b203-21c69803887f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:45.372912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe49e3-a2a2-47a3-b203-21c69803887f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:45.373180 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe49e3-a2a2-47a3-b203-21c69803887f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:45.373862 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe49e3-a2a2-47a3-b203-21c69803887f None admin] RBAC: Adding query filter params (interface=public, service_id=385206425aa44865804bb4e243f0f8b9, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:45.374165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe49e3-a2a2-47a3-b203-21c69803887f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:45.375539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe49e3-a2a2-47a3-b203-21c69803887f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:45.380878 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 644/1300] 10.222.0.39 () {56 vars in 1251 bytes} [Mon Aug 8 02:16:45 2022] GET /identity/v3/endpoints?service_id=385206425aa44865804bb4e243f0f8b9&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:45.386529 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df21bf7-1768-40c5-ae4f-b691d293ecf1 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:45.466741 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:45.485761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df21bf7-1768-40c5-ae4f-b691d293ecf1 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:45.517902 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735f3f5-8281-4224-9924-d35b5135a7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:45.518540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735f3f5-8281-4224-9924-d35b5135a7ed None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:45.518815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735f3f5-8281-4224-9924-d35b5135a7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:45.519083 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735f3f5-8281-4224-9924-d35b5135a7ed None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:45.519680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735f3f5-8281-4224-9924-d35b5135a7ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:45.521044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735f3f5-8281-4224-9924-d35b5135a7ed None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:45.527596 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 657/1301] 10.222.0.39 () {56 vars in 1087 bytes} [Mon Aug 8 02:16:45 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:16:47.516156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a8014-9b05-404e-b3b0-56b81bf4a278 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:47.517097 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a8014-9b05-404e-b3b0-56b81bf4a278 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:47.517211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a8014-9b05-404e-b3b0-56b81bf4a278 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:47.518540 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 645/1302] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:47.523518 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf773886-d8f7-4bcd-8b92-ba00ecc2ff0a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:47.523888 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf773886-d8f7-4bcd-8b92-ba00ecc2ff0a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:47.524068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf773886-d8f7-4bcd-8b92-ba00ecc2ff0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:47.542042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-cf773886-d8f7-4bcd-8b92-ba00ecc2ff0a None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:47.543266 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cf773886-d8f7-4bcd-8b92-ba00ecc2ff0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:47.606902 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 658/1303] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:47 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:47.629589 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a8014-9b05-404e-b3b0-56b81bf4a278 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:47.724904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:47.731715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a8014-9b05-404e-b3b0-56b81bf4a278 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:47.768738 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e2483-f5b0-4251-a1e4-5221e24e3308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:47.769353 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e2483-f5b0-4251-a1e4-5221e24e3308 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:47.769632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e2483-f5b0-4251-a1e4-5221e24e3308 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:47.769900 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e2483-f5b0-4251-a1e4-5221e24e3308 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:47.770502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e2483-f5b0-4251-a1e4-5221e24e3308 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:47.771861 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e2483-f5b0-4251-a1e4-5221e24e3308 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:47.780324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-598e2483-f5b0-4251-a1e4-5221e24e3308 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:16:47.780324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:47.780324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:16:47.780324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:16:47.780324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:16:47.780324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:16:47.780324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:16:47.780324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:16:47.780324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:16:47.780324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:47.780324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:16:47.780324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:47.780324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:47.780324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:47.780324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:47.780324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:47.780324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:47.781485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:47.781485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:47.781485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:47.781485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:47.781485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:47.781485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:47.781485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:16:47.781485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:16:47.781485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:16:47.781485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:16:47.781485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:47.781485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:47.781485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:47.781485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:47.781485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:47.781485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:47.782658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:47.782658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:47.782658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:47.782658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:47.782658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:47.782658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:47.782658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:47.782658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:47.782658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:47.782658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:47.782658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:16:47.782658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:16:47.782658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:16:47.782658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 08 02:16:47.782658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 646/1304] 10.222.0.39 () {56 vars in 1117 bytes} [Mon Aug 8 02:16:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:47.787416 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf773886-d8f7-4bcd-8b92-ba00ecc2ff0a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:47.860911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:47.884054 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf773886-d8f7-4bcd-8b92-ba00ecc2ff0a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:47.917313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ceba4-b6d8-4253-ae8e-0ab7c2453767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:47.917899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ceba4-b6d8-4253-ae8e-0ab7c2453767 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:47.918174 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ceba4-b6d8-4253-ae8e-0ab7c2453767 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:47.918473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ceba4-b6d8-4253-ae8e-0ab7c2453767 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:47.919094 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ceba4-b6d8-4253-ae8e-0ab7c2453767 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:47.919393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ceba4-b6d8-4253-ae8e-0ab7c2453767 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:47.920749 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ceba4-b6d8-4253-ae8e-0ab7c2453767 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:47.925662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 659/1305] 10.222.0.39 () {56 vars in 1116 bytes} [Mon Aug 8 02:16:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:47.931274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e2483-f5b0-4251-a1e4-5221e24e3308 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:48.027739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e2483-f5b0-4251-a1e4-5221e24e3308 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:48.059880 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c1ad4-84bc-4c3d-b1d3-e42167cd1009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:48.060458 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c1ad4-84bc-4c3d-b1d3-e42167cd1009 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:48.060736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c1ad4-84bc-4c3d-b1d3-e42167cd1009 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:48.061003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c1ad4-84bc-4c3d-b1d3-e42167cd1009 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:48.061639 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c1ad4-84bc-4c3d-b1d3-e42167cd1009 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:48.063011 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c1ad4-84bc-4c3d-b1d3-e42167cd1009 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:48.084812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 647/1306] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:16:47 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:16:57.923091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=30 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:58.080869 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:50.449269 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fba6a4-85bf-4652-be38-1fb82f0c018f None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:50.449711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fba6a4-85bf-4652-be38-1fb82f0c018f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:50.450119 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fba6a4-85bf-4652-be38-1fb82f0c018f None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:50.459058 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79c783-bf56-4a6a-aa63-a1aa701dcddc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:50.459489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79c783-bf56-4a6a-aa63-a1aa701dcddc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:50.459796 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79c783-bf56-4a6a-aa63-a1aa701dcddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:50.462786 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 660/1307] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:17:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:50.553426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5d79c783-bf56-4a6a-aa63-a1aa701dcddc None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:50.555000 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5d79c783-bf56-4a6a-aa63-a1aa701dcddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:50.701767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 648/1308] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:17:50.762215 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a79de-c9ba-478b-8dc9-b5d152a9fc46 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:50.762683 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a79de-c9ba-478b-8dc9-b5d152a9fc46 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:50.763032 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a79de-c9ba-478b-8dc9-b5d152a9fc46 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:50.764799 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 661/1309] 10.222.0.39 () {54 vars in 965 bytes} [Mon Aug 8 02:17:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:50.772507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a569e-90a0-48a5-940e-0dfd7cf47f2b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:50.773304 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a569e-90a0-48a5-940e-0dfd7cf47f2b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:50.773781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a569e-90a0-48a5-940e-0dfd7cf47f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:50.862502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f59a569e-90a0-48a5-940e-0dfd7cf47f2b None None] MFA Rules not processed for user `78506fea1c024b169a4b7ee86116658c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:50.863818 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f59a569e-90a0-48a5-940e-0dfd7cf47f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:50.885211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:50.943039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 649/1310] 10.222.0.39 () {58 vars in 1062 bytes} [Mon Aug 8 02:17:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:17:50.952112 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a79de-c9ba-478b-8dc9-b5d152a9fc46 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:50.953839 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-542a79de-c9ba-478b-8dc9-b5d152a9fc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:51.074761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a79de-c9ba-478b-8dc9-b5d152a9fc46 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:51.153709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5290f0-a091-4c6c-876d-e169074e485c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:51.154457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5290f0-a091-4c6c-876d-e169074e485c service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:51.154727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5290f0-a091-4c6c-876d-e169074e485c service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:51.155679 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5290f0-a091-4c6c-876d-e169074e485c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:51.157283 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3a5290f0-a091-4c6c-876d-e169074e485c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:51.215349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:51.273949 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5290f0-a091-4c6c-876d-e169074e485c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:51.275517 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5290f0-a091-4c6c-876d-e169074e485c service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:51.295364 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 662/1311] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:17:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:17:53.555654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af38473-bc17-49c4-840a-6eeb82dad108 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:53.556193 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af38473-bc17-49c4-840a-6eeb82dad108 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:53.556193 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af38473-bc17-49c4-840a-6eeb82dad108 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:53.557522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 650/1312] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:53.564631 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320525be-4d5b-40f8-91b8-fa66a32b2643 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:53.565088 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320525be-4d5b-40f8-91b8-fa66a32b2643 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:53.565184 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320525be-4d5b-40f8-91b8-fa66a32b2643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:53.646576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-320525be-4d5b-40f8-91b8-fa66a32b2643 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:53.647858 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-320525be-4d5b-40f8-91b8-fa66a32b2643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:53.759972 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 663/1313] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:17:53.813012 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc82aa4-a1b2-462c-b1d4-2e229f4aa0a5 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:53.813433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc82aa4-a1b2-462c-b1d4-2e229f4aa0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:53.813811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc82aa4-a1b2-462c-b1d4-2e229f4aa0a5 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:53.815317 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 651/1314] 10.222.0.39 () {54 vars in 965 bytes} [Mon Aug 8 02:17:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:53.821474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ee038-791b-475b-82d4-d8bd5e3c5a4e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:53.821832 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ee038-791b-475b-82d4-d8bd5e3c5a4e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:53.822215 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ee038-791b-475b-82d4-d8bd5e3c5a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:53.835040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:53.902468 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-aa6ee038-791b-475b-82d4-d8bd5e3c5a4e None None] MFA Rules not processed for user `78506fea1c024b169a4b7ee86116658c`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:53.903852 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-aa6ee038-791b-475b-82d4-d8bd5e3c5a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:53.985694 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 664/1315] 10.222.0.39 () {58 vars in 1062 bytes} [Mon Aug 8 02:17:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:17:53.994385 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc82aa4-a1b2-462c-b1d4-2e229f4aa0a5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:53.996030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-edc82aa4-a1b2-462c-b1d4-2e229f4aa0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:54.110139 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc82aa4-a1b2-462c-b1d4-2e229f4aa0a5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:54.185462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:54.188278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83211827-c2b2-4101-9813-c5fdda69f0ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:54.188881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83211827-c2b2-4101-9813-c5fdda69f0ba service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:54.189158 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83211827-c2b2-4101-9813-c5fdda69f0ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:54.189429 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83211827-c2b2-4101-9813-c5fdda69f0ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:54.191057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-83211827-c2b2-4101-9813-c5fdda69f0ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:54.298975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83211827-c2b2-4101-9813-c5fdda69f0ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:54.300330 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83211827-c2b2-4101-9813-c5fdda69f0ba service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:54.318884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 652/1316] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:17:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:17:56.891946 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba8224-3d81-4954-b827-fc5fc56e46aa None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:56.892504 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba8224-3d81-4954-b827-fc5fc56e46aa None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:56.892504 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba8224-3d81-4954-b827-fc5fc56e46aa None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:56.893833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 665/1317] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:56.899066 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cee559-1b14-4331-99d9-2bbd463080ef None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:56.899505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cee559-1b14-4331-99d9-2bbd463080ef None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:56.899601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cee559-1b14-4331-99d9-2bbd463080ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:56.964145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-28cee559-1b14-4331-99d9-2bbd463080ef None None] MFA Rules not processed for user `907bde105eaf466980f8a555198f3c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:56.965359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-28cee559-1b14-4331-99d9-2bbd463080ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:57.015983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:57.061795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 653/1318] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:56 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:17:57.100826 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba8224-3d81-4954-b827-fc5fc56e46aa None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:57.102378 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c3ba8224-3d81-4954-b827-fc5fc56e46aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:57.203770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba8224-3d81-4954-b827-fc5fc56e46aa None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:57.206694 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:57.275718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed65cd-8048-4deb-a179-75c4739842f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:57.276312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed65cd-8048-4deb-a179-75c4739842f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:57.276603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed65cd-8048-4deb-a179-75c4739842f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:57.276851 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed65cd-8048-4deb-a179-75c4739842f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:57.278452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f9ed65cd-8048-4deb-a179-75c4739842f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:57.383983 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed65cd-8048-4deb-a179-75c4739842f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:57.385394 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed65cd-8048-4deb-a179-75c4739842f4 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:57.404204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 666/1319] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:17:57 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:17:59.989394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e54d1c-4482-4d6f-9423-d7d4f40f3609 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:59.989927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e54d1c-4482-4d6f-9423-d7d4f40f3609 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:59.989927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e54d1c-4482-4d6f-9423-d7d4f40f3609 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:59.991334 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 654/1320] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:59.996363 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619214c-56fa-4083-b329-3cea9959863e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:59.996806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619214c-56fa-4083-b329-3cea9959863e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:59.996911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619214c-56fa-4083-b329-3cea9959863e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:00.036438 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:00.061965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-2619214c-56fa-4083-b329-3cea9959863e None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:00.063830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2619214c-56fa-4083-b329-3cea9959863e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:00.160477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 667/1321] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:00.199574 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e54d1c-4482-4d6f-9423-d7d4f40f3609 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:00.201117 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d8e54d1c-4482-4d6f-9423-d7d4f40f3609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:00.308237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e54d1c-4482-4d6f-9423-d7d4f40f3609 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:00.380041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a930389-773e-4198-a14a-8f391615df3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:00.380639 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a930389-773e-4198-a14a-8f391615df3f service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:00.380912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a930389-773e-4198-a14a-8f391615df3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:00.381181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a930389-773e-4198-a14a-8f391615df3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:00.382782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7a930389-773e-4198-a14a-8f391615df3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:00.389939 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:00.491813 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a930389-773e-4198-a14a-8f391615df3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:00.493235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a930389-773e-4198-a14a-8f391615df3f service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:00.510946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 655/1322] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:03.167540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4aacc4-d2b2-4e80-bc44-23f2897c6bcb None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:03.168093 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4aacc4-d2b2-4e80-bc44-23f2897c6bcb None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:03.168093 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4aacc4-d2b2-4e80-bc44-23f2897c6bcb None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:03.169416 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 668/1323] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:03.175139 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2d428-b4c5-4a74-92ba-cbe1f9fc2267 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:03.175560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2d428-b4c5-4a74-92ba-cbe1f9fc2267 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:03.175793 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2d428-b4c5-4a74-92ba-cbe1f9fc2267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:03.240059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-eff2d428-b4c5-4a74-92ba-cbe1f9fc2267 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:03.241240 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-eff2d428-b4c5-4a74-92ba-cbe1f9fc2267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:03.302192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:03.344764 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 656/1324] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:03.381621 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4aacc4-d2b2-4e80-bc44-23f2897c6bcb None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:03.383248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5d4aacc4-d2b2-4e80-bc44-23f2897c6bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:03.495074 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4aacc4-d2b2-4e80-bc44-23f2897c6bcb None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:03.507253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:03.569464 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185aae7-1e5f-4495-885d-c33820a5f48d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:03.570070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185aae7-1e5f-4495-885d-c33820a5f48d service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:03.570372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185aae7-1e5f-4495-885d-c33820a5f48d service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:03.570639 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185aae7-1e5f-4495-885d-c33820a5f48d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:03.572221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5185aae7-1e5f-4495-885d-c33820a5f48d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:03.679016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185aae7-1e5f-4495-885d-c33820a5f48d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:03.680365 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185aae7-1e5f-4495-885d-c33820a5f48d service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:03.698500 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 669/1325] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:05.748927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddff25c-0588-407c-8428-b0dea9ccec89 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:05.749446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddff25c-0588-407c-8428-b0dea9ccec89 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:05.749446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddff25c-0588-407c-8428-b0dea9ccec89 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:05.750918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 657/1326] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:05.755840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e5305-c39b-4102-b0ae-e0878d376d66 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:05.756229 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e5305-c39b-4102-b0ae-e0878d376d66 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:05.756393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e5305-c39b-4102-b0ae-e0878d376d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:05.805363 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:05.823670 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-106e5305-c39b-4102-b0ae-e0878d376d66 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:05.824869 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-106e5305-c39b-4102-b0ae-e0878d376d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:05.919664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 670/1327] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:05.955514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddff25c-0588-407c-8428-b0dea9ccec89 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:05.957051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5ddff25c-0588-407c-8428-b0dea9ccec89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:06.063608 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddff25c-0588-407c-8428-b0dea9ccec89 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:06.142049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5facf-adf0-4a73-ab73-af3cfdf72647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:06.142694 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5facf-adf0-4a73-ab73-af3cfdf72647 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:06.142971 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5facf-adf0-4a73-ab73-af3cfdf72647 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:06.143236 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5facf-adf0-4a73-ab73-af3cfdf72647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:06.144923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f9c5facf-adf0-4a73-ab73-af3cfdf72647 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:06.162431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:06.254452 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5facf-adf0-4a73-ab73-af3cfdf72647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:06.255897 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5facf-adf0-4a73-ab73-af3cfdf72647 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:06.274557 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 658/1328] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:08.326772 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdd837-5227-47b8-ac5c-ff612b096787 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:08.327317 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdd837-5227-47b8-ac5c-ff612b096787 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:08.327317 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdd837-5227-47b8-ac5c-ff612b096787 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:08.328595 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 671/1329] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:08.333871 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa08c9-e131-4beb-afca-e44dbf418f4f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:08.334274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa08c9-e131-4beb-afca-e44dbf418f4f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:08.334452 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa08c9-e131-4beb-afca-e44dbf418f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:08.399080 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6eaa08c9-e131-4beb-afca-e44dbf418f4f None None] MFA Rules not processed for user `907bde105eaf466980f8a555198f3c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:08.400307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6eaa08c9-e131-4beb-afca-e44dbf418f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:08.479300 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:08.512883 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 659/1330] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:08 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:08.552639 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefdd837-5227-47b8-ac5c-ff612b096787 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:08.554190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-eefdd837-5227-47b8-ac5c-ff612b096787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:08.656891 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefdd837-5227-47b8-ac5c-ff612b096787 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:08.676903 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:08.730440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e499a-2715-4378-a2a9-927d2090c7ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:08.731044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e499a-2715-4378-a2a9-927d2090c7ea service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:08.731321 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e499a-2715-4378-a2a9-927d2090c7ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:08.731586 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e499a-2715-4378-a2a9-927d2090c7ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:08.733182 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d27e499a-2715-4378-a2a9-927d2090c7ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:08.844026 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e499a-2715-4378-a2a9-927d2090c7ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:08.845487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e499a-2715-4378-a2a9-927d2090c7ea service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:08.863688 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 672/1331] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:08 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:11.116364 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c78412-e037-47ae-bfee-3c7c32d348a6 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:11.116932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c78412-e037-47ae-bfee-3c7c32d348a6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:11.116932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c78412-e037-47ae-bfee-3c7c32d348a6 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:11.118241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 660/1332] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:11.123363 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27179815-89dd-40dc-94a9-e08fbcbd0a42 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:11.123735 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27179815-89dd-40dc-94a9-e08fbcbd0a42 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:11.123945 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27179815-89dd-40dc-94a9-e08fbcbd0a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:11.186679 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:11.189327 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-27179815-89dd-40dc-94a9-e08fbcbd0a42 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:11.191080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-27179815-89dd-40dc-94a9-e08fbcbd0a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:11.286076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 673/1333] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:11.321104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c78412-e037-47ae-bfee-3c7c32d348a6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:11.322679 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-06c78412-e037-47ae-bfee-3c7c32d348a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:11.423735 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c78412-e037-47ae-bfee-3c7c32d348a6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:11.494954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c5f16-fe8f-483a-aa00-1ac3b0a20e7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:11.495547 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c5f16-fe8f-483a-aa00-1ac3b0a20e7b service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:11.495824 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c5f16-fe8f-483a-aa00-1ac3b0a20e7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:11.496091 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c5f16-fe8f-483a-aa00-1ac3b0a20e7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:11.497670 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-993c5f16-fe8f-483a-aa00-1ac3b0a20e7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:11.525300 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:11.605422 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c5f16-fe8f-483a-aa00-1ac3b0a20e7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:11.606882 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c5f16-fe8f-483a-aa00-1ac3b0a20e7b service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:11.624634 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 661/1334] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:13.646414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4b3d4-24eb-41bd-88ce-b2b01f2733f4 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:13.646963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4b3d4-24eb-41bd-88ce-b2b01f2733f4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:13.646963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4b3d4-24eb-41bd-88ce-b2b01f2733f4 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:13.648292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 674/1335] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:13.653374 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23414aa-53d7-4080-a0a5-b74c8da32428 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:13.653767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23414aa-53d7-4080-a0a5-b74c8da32428 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:13.653916 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23414aa-53d7-4080-a0a5-b74c8da32428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:13.718273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d23414aa-53d7-4080-a0a5-b74c8da32428 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:13.719394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d23414aa-53d7-4080-a0a5-b74c8da32428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:13.792580 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:13.815546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 662/1336] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:13.854860 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4b3d4-24eb-41bd-88ce-b2b01f2733f4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:13.856387 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d4c4b3d4-24eb-41bd-88ce-b2b01f2733f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:13.957649 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4b3d4-24eb-41bd-88ce-b2b01f2733f4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:13.985771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:14.030021 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c2a1d-1ee5-471a-89e1-4cf03400a652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:14.031193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c2a1d-1ee5-471a-89e1-4cf03400a652 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:14.031463 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c2a1d-1ee5-471a-89e1-4cf03400a652 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:14.031726 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c2a1d-1ee5-471a-89e1-4cf03400a652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:14.033270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5c4c2a1d-1ee5-471a-89e1-4cf03400a652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:14.138437 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4c2a1d-1ee5-471a-89e1-4cf03400a652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:14.140363 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4c2a1d-1ee5-471a-89e1-4cf03400a652 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:14.158576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 675/1337] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:16.821411 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf146d-d829-42ce-b94c-d59d1bfe4f87 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:16.821943 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf146d-d829-42ce-b94c-d59d1bfe4f87 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:16.821943 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf146d-d829-42ce-b94c-d59d1bfe4f87 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:16.823680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 663/1338] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:16.828909 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e89c12-5ebc-4999-b7b9-f64cf8d3920b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:16.829352 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e89c12-5ebc-4999-b7b9-f64cf8d3920b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:16.829447 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e89c12-5ebc-4999-b7b9-f64cf8d3920b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:16.894552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-e6e89c12-5ebc-4999-b7b9-f64cf8d3920b None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:16.895757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e6e89c12-5ebc-4999-b7b9-f64cf8d3920b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:16.903877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:16.990376 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 676/1339] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:17.026830 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf146d-d829-42ce-b94c-d59d1bfe4f87 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:17.028344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4aaf146d-d829-42ce-b94c-d59d1bfe4f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:17.130996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf146d-d829-42ce-b94c-d59d1bfe4f87 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:17.204556 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054001f-2f18-4358-a1be-8738e1d725b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:17.205153 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054001f-2f18-4358-a1be-8738e1d725b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:17.205428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054001f-2f18-4358-a1be-8738e1d725b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:17.205695 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054001f-2f18-4358-a1be-8738e1d725b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:17.207329 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e054001f-2f18-4358-a1be-8738e1d725b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:17.241879 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:17.315148 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054001f-2f18-4358-a1be-8738e1d725b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:17.316589 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054001f-2f18-4358-a1be-8738e1d725b9 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:17.336775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 664/1340] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:21.234007 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e719ff5-4d27-4218-b800-f1c60165747f None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:21.234627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e719ff5-4d27-4218-b800-f1c60165747f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:21.234627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e719ff5-4d27-4218-b800-f1c60165747f None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:21.235904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 677/1341] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:18:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:21.240982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24facee2-aa81-4f94-83b7-44a2012a8c88 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:21.241402 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24facee2-aa81-4f94-83b7-44a2012a8c88 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:21.241537 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24facee2-aa81-4f94-83b7-44a2012a8c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:21.306838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-24facee2-aa81-4f94-83b7-44a2012a8c88 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:21.307978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-24facee2-aa81-4f94-83b7-44a2012a8c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:21.392948 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:21.404767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 665/1342] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:18:21.445209 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e719ff5-4d27-4218-b800-f1c60165747f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:21.446856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4e719ff5-4d27-4218-b800-f1c60165747f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:21.551221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e719ff5-4d27-4218-b800-f1c60165747f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:21.588338 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:21.624509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73d4ac-d6e6-442c-99e5-8b8668a980de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:21.625108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73d4ac-d6e6-442c-99e5-8b8668a980de service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:21.625380 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73d4ac-d6e6-442c-99e5-8b8668a980de service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:21.625648 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73d4ac-d6e6-442c-99e5-8b8668a980de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:21.627247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5b73d4ac-d6e6-442c-99e5-8b8668a980de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:21.733856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73d4ac-d6e6-442c-99e5-8b8668a980de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:21.735306 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73d4ac-d6e6-442c-99e5-8b8668a980de service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:21.752854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 678/1343] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:24.408038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f5343-fe00-476a-afe3-7e82d051c959 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:24.408578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f5343-fe00-476a-afe3-7e82d051c959 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:24.408578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f5343-fe00-476a-afe3-7e82d051c959 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:24.409868 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 666/1344] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:24.415441 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaa09b-5e5e-41eb-9593-9e2b2eab3655 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:24.415960 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaa09b-5e5e-41eb-9593-9e2b2eab3655 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:24.416054 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaa09b-5e5e-41eb-9593-9e2b2eab3655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:24.487904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-bacaa09b-5e5e-41eb-9593-9e2b2eab3655 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:24.489176 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bacaa09b-5e5e-41eb-9593-9e2b2eab3655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:24.507420 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:24.586657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 679/1345] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:24.623834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f5343-fe00-476a-afe3-7e82d051c959 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:24.625349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7c4f5343-fe00-476a-afe3-7e82d051c959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:24.728083 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f5343-fe00-476a-afe3-7e82d051c959 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:24.802276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910b5d9-c12a-4f75-b844-e93c0915b774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:24.802878 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910b5d9-c12a-4f75-b844-e93c0915b774 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:24.803152 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910b5d9-c12a-4f75-b844-e93c0915b774 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:24.803419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910b5d9-c12a-4f75-b844-e93c0915b774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:24.804985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3910b5d9-c12a-4f75-b844-e93c0915b774 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:24.848696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:24.916379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910b5d9-c12a-4f75-b844-e93c0915b774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:24.917812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910b5d9-c12a-4f75-b844-e93c0915b774 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:24.937895 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 667/1346] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:29.303707 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ef000-39c6-472e-8a5b-7b4d1a08a021 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:29.304260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ef000-39c6-472e-8a5b-7b4d1a08a021 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:29.304355 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ef000-39c6-472e-8a5b-7b4d1a08a021 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:29.305923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 680/1347] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:29.311380 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68335e-c640-49bc-9395-f96cd644a456 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:29.311878 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68335e-c640-49bc-9395-f96cd644a456 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:29.311971 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68335e-c640-49bc-9395-f96cd644a456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:29.381146 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2b68335e-c640-49bc-9395-f96cd644a456 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:29.382489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2b68335e-c640-49bc-9395-f96cd644a456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:29.480838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:29.483748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 668/1348] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:29.522654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ef000-39c6-472e-8a5b-7b4d1a08a021 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:29.524288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1c1ef000-39c6-472e-8a5b-7b4d1a08a021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:29.631017 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ef000-39c6-472e-8a5b-7b4d1a08a021 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:29.678930 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:29.706773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6896d50-fe96-477b-a023-11e643b331c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:29.707381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6896d50-fe96-477b-a023-11e643b331c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:29.707657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6896d50-fe96-477b-a023-11e643b331c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:29.707925 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6896d50-fe96-477b-a023-11e643b331c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:29.709611 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b6896d50-fe96-477b-a023-11e643b331c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:29.821950 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6896d50-fe96-477b-a023-11e643b331c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:29.823408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6896d50-fe96-477b-a023-11e643b331c1 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:29.841022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 681/1349] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:34.782310 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0727d-5c58-4281-ae8a-66f8e04de2b4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:34.782310 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0727d-5c58-4281-ae8a-66f8e04de2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:34.782310 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0727d-5c58-4281-ae8a-66f8e04de2b4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:34.795704 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc7cfb-4bad-47dd-8a8a-1943d3200c70 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:34.796198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc7cfb-4bad-47dd-8a8a-1943d3200c70 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:34.796632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc7cfb-4bad-47dd-8a8a-1943d3200c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:34.803910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 669/1350] 10.222.0.39 () {54 vars in 921 bytes} [Mon Aug 8 02:18:34 2022] GET /identity => generated 270 bytes in 27 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:34.916289 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-1ffc7cfb-4bad-47dd-8a8a-1943d3200c70 None None] MFA Rules not processed for user `f62745b92a864480a9438de581e1d05e`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:34.917497 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1ffc7cfb-4bad-47dd-8a8a-1943d3200c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:34.937068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:35.031426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 682/1351] 10.222.0.39 () {58 vars in 1018 bytes} [Mon Aug 8 02:18:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:39.002764 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e894e-39a5-42ab-99b2-79552f9bc381 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:39.003271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e894e-39a5-42ab-99b2-79552f9bc381 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:39.003400 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e894e-39a5-42ab-99b2-79552f9bc381 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:39.004833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 670/1352] 10.222.0.39 () {54 vars in 921 bytes} [Mon Aug 8 02:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:39.009641 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599ab2a-59f4-4866-90f7-09199f625e05 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:39.010092 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599ab2a-59f4-4866-90f7-09199f625e05 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:39.010439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599ab2a-59f4-4866-90f7-09199f625e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:39.092229 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6599ab2a-59f4-4866-90f7-09199f625e05 None None] MFA Rules not processed for user `f62745b92a864480a9438de581e1d05e`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:39.094062 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6599ab2a-59f4-4866-90f7-09199f625e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:39.168572 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:39.208669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 683/1353] 10.222.0.39 () {58 vars in 1018 bytes} [Mon Aug 8 02:18:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:40.708612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477f6fe-7591-4ef2-9379-4aff36ba3654 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:40.709115 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477f6fe-7591-4ef2-9379-4aff36ba3654 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:40.709115 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477f6fe-7591-4ef2-9379-4aff36ba3654 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:40.710430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 671/1354] 10.222.0.39 () {54 vars in 921 bytes} [Mon Aug 8 02:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:40.715539 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75092b-71ad-490b-a58b-c33f316d2f1e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:40.715973 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75092b-71ad-490b-a58b-c33f316d2f1e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:40.716224 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75092b-71ad-490b-a58b-c33f316d2f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:40.792889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-bf75092b-71ad-490b-a58b-c33f316d2f1e None None] MFA Rules not processed for user `f62745b92a864480a9438de581e1d05e`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:40.794104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bf75092b-71ad-490b-a58b-c33f316d2f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:40.902426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 684/1355] 10.222.0.39 () {58 vars in 1018 bytes} [Mon Aug 8 02:18:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:18:44.163712 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea8675-675f-4670-a1c7-d44a4be26cb0 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:44.164555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea8675-675f-4670-a1c7-d44a4be26cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:44.165010 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea8675-675f-4670-a1c7-d44a4be26cb0 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:44.167477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 672/1356] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:18:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:44.173011 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0141501-a594-46a6-83fd-78cb7b5f6099 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:44.173475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0141501-a594-46a6-83fd-78cb7b5f6099 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:44.173583 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0141501-a594-46a6-83fd-78cb7b5f6099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:44.228136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:44.265152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c0141501-a594-46a6-83fd-78cb7b5f6099 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:44.266782 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c0141501-a594-46a6-83fd-78cb7b5f6099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:44.369895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 685/1357] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:44.430702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68875e-f13f-48cb-a6ac-b9abe3fa8554 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:44.431091 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68875e-f13f-48cb-a6ac-b9abe3fa8554 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:44.431608 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68875e-f13f-48cb-a6ac-b9abe3fa8554 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:44.433447 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 673/1358] 10.222.0.39 () {54 vars in 962 bytes} [Mon Aug 8 02:18:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:44.447949 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d93df-0a67-4b82-aba7-27baf47045dc None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:44.448669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d93df-0a67-4b82-aba7-27baf47045dc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:44.449220 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d93df-0a67-4b82-aba7-27baf47045dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:44.534473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a65d93df-0a67-4b82-aba7-27baf47045dc None None] MFA Rules not processed for user `c4238566a9c248ffbfa314f47f02670c`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:44.535668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a65d93df-0a67-4b82-aba7-27baf47045dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:44.586059 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:44.656300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 686/1359] 10.222.0.39 () {58 vars in 1059 bytes} [Mon Aug 8 02:18:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:44.664413 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68875e-f13f-48cb-a6ac-b9abe3fa8554 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:44.666130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6e68875e-f13f-48cb-a6ac-b9abe3fa8554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:44.817256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68875e-f13f-48cb-a6ac-b9abe3fa8554 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:44.892069 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:44.933587 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940213b-c7fb-4a73-94c8-a0105dfff13b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:44.934113 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940213b-c7fb-4a73-94c8-a0105dfff13b service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:44.934425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940213b-c7fb-4a73-94c8-a0105dfff13b service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:44.934695 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940213b-c7fb-4a73-94c8-a0105dfff13b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:44.937172 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4940213b-c7fb-4a73-94c8-a0105dfff13b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:45.030415 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940213b-c7fb-4a73-94c8-a0105dfff13b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:45.032083 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940213b-c7fb-4a73-94c8-a0105dfff13b service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:45.051286 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 674/1360] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:47.267614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7470bf6-4bfd-4215-aae4-73758560d8ae None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:47.268141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7470bf6-4bfd-4215-aae4-73758560d8ae None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:47.268141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7470bf6-4bfd-4215-aae4-73758560d8ae None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:47.269642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 687/1361] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:47.274564 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561279d3-282d-40fe-a815-4d9227f80203 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:47.274962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561279d3-282d-40fe-a815-4d9227f80203 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:47.275121 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561279d3-282d-40fe-a815-4d9227f80203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:47.348551 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-561279d3-282d-40fe-a815-4d9227f80203 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:47.349678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-561279d3-282d-40fe-a815-4d9227f80203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:47.389800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:47.463087 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 675/1362] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:47.508586 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7470bf6-4bfd-4215-aae4-73758560d8ae None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:47.510395 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b7470bf6-4bfd-4215-aae4-73758560d8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:47.684650 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7470bf6-4bfd-4215-aae4-73758560d8ae None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:47.695549 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:47.827263 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c880ba9-3dfd-4803-a06f-45bdde2b4472 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:47.827865 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c880ba9-3dfd-4803-a06f-45bdde2b4472 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:47.828138 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c880ba9-3dfd-4803-a06f-45bdde2b4472 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:47.828405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c880ba9-3dfd-4803-a06f-45bdde2b4472 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:47.830073 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3c880ba9-3dfd-4803-a06f-45bdde2b4472 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:47.935362 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c880ba9-3dfd-4803-a06f-45bdde2b4472 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:47.937618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c880ba9-3dfd-4803-a06f-45bdde2b4472 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:47.956454 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 688/1363] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:50.287877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3542c-c956-4d4e-aa39-6224e245b187 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:50.288397 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3542c-c956-4d4e-aa39-6224e245b187 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:50.288645 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3542c-c956-4d4e-aa39-6224e245b187 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:50.290133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 676/1364] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:50.296031 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8554f5c-93d2-4d80-bb7e-91fcd4bb6875 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:50.296481 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8554f5c-93d2-4d80-bb7e-91fcd4bb6875 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:50.296561 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8554f5c-93d2-4d80-bb7e-91fcd4bb6875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:50.313748 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:50.376784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c8554f5c-93d2-4d80-bb7e-91fcd4bb6875 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:50.377940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c8554f5c-93d2-4d80-bb7e-91fcd4bb6875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:50.497804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 689/1365] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:50.545899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab774f38-3161-4c5a-93c4-089a83311681 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:50.546492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab774f38-3161-4c5a-93c4-089a83311681 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:50.546940 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab774f38-3161-4c5a-93c4-089a83311681 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:50.548441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 677/1366] 10.222.0.39 () {54 vars in 962 bytes} [Mon Aug 8 02:18:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:50.553938 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d875c56-30da-42de-9934-9e1a20de13d4 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:50.554225 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d875c56-30da-42de-9934-9e1a20de13d4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:50.554540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d875c56-30da-42de-9934-9e1a20de13d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:50.640922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-9d875c56-30da-42de-9934-9e1a20de13d4 None None] MFA Rules not processed for user `c4238566a9c248ffbfa314f47f02670c`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:50.642468 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9d875c56-30da-42de-9934-9e1a20de13d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:50.680179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:50.785615 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 690/1367] 10.222.0.39 () {58 vars in 1059 bytes} [Mon Aug 8 02:18:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:50.794969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab774f38-3161-4c5a-93c4-089a83311681 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:50.797129 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ab774f38-3161-4c5a-93c4-089a83311681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:50.958460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab774f38-3161-4c5a-93c4-089a83311681 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:50.990323 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:51.097093 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a90b5a-dcae-40e3-89f7-3b2ca96ec3e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:51.097699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a90b5a-dcae-40e3-89f7-3b2ca96ec3e0 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:51.097978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a90b5a-dcae-40e3-89f7-3b2ca96ec3e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:51.098339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a90b5a-dcae-40e3-89f7-3b2ca96ec3e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:51.099963 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e8a90b5a-dcae-40e3-89f7-3b2ca96ec3e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:51.200338 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a90b5a-dcae-40e3-89f7-3b2ca96ec3e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:51.202777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a90b5a-dcae-40e3-89f7-3b2ca96ec3e0 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:51.223181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 678/1368] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:52.292592 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ec85e-5927-4ac6-897e-219512d2470b None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:52.293279 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ec85e-5927-4ac6-897e-219512d2470b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:52.293279 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ec85e-5927-4ac6-897e-219512d2470b None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:52.294669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 691/1369] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:52.300105 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf80ebe-873a-45db-83a5-f40e4a9fff76 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:52.300559 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf80ebe-873a-45db-83a5-f40e4a9fff76 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:52.300657 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf80ebe-873a-45db-83a5-f40e4a9fff76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:52.332996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:52.370042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fcf80ebe-873a-45db-83a5-f40e4a9fff76 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:52.371295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fcf80ebe-873a-45db-83a5-f40e4a9fff76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:52.482419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 679/1370] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:52.817258 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6affc-d8a0-4ec3-aec0-0cc7ff4da5fc None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:52.817548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6affc-d8a0-4ec3-aec0-0cc7ff4da5fc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:52.817825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6affc-d8a0-4ec3-aec0-0cc7ff4da5fc None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:52.819435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 692/1371] 10.222.0.39 () {54 vars in 964 bytes} [Mon Aug 8 02:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:52.824360 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd823c-ac0a-44f1-89f4-81bf18d4ca9b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:52.824717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd823c-ac0a-44f1-89f4-81bf18d4ca9b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:52.825064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd823c-ac0a-44f1-89f4-81bf18d4ca9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:52.907332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-74cd823c-ac0a-44f1-89f4-81bf18d4ca9b None None] MFA Rules not processed for user `2812e59d0d9948879a465fe3b5b421e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:52.908548 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-74cd823c-ac0a-44f1-89f4-81bf18d4ca9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:52.958206 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:52.995550 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 680/1372] 10.222.0.39 () {58 vars in 1061 bytes} [Mon Aug 8 02:18:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:53.003151 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6affc-d8a0-4ec3-aec0-0cc7ff4da5fc None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:53.004845 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bcb6affc-d8a0-4ec3-aec0-0cc7ff4da5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:53.116080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6affc-d8a0-4ec3-aec0-0cc7ff4da5fc None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:53.157378 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:53.192931 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a9645-6235-4ae3-8468-44d6d635c413 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:53.193536 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a9645-6235-4ae3-8468-44d6d635c413 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:53.193817 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a9645-6235-4ae3-8468-44d6d635c413 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:53.194091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a9645-6235-4ae3-8468-44d6d635c413 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:53.195749 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f85a9645-6235-4ae3-8468-44d6d635c413 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:53.311120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba02a8f-dd1a-40e5-8215-4f4441b90eb2 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:53.311616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba02a8f-dd1a-40e5-8215-4f4441b90eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:53.311616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba02a8f-dd1a-40e5-8215-4f4441b90eb2 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:53.313290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 681/1373] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:53.316174 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a9645-6235-4ae3-8468-44d6d635c413 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:53.317620 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a9645-6235-4ae3-8468-44d6d635c413 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:53.317911 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd63a8-0302-4eab-8f93-b5508ad5abc7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:53.318197 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd63a8-0302-4eab-8f93-b5508ad5abc7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:53.318502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd63a8-0302-4eab-8f93-b5508ad5abc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:53.337048 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 693/1374] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:53.391889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bcfd63a8-0302-4eab-8f93-b5508ad5abc7 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:53.393068 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bcfd63a8-0302-4eab-8f93-b5508ad5abc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:53.517730 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 682/1375] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:18:53.563397 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a9645-6235-4ae3-8468-44d6d635c413 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:53.564667 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f85a9645-6235-4ae3-8468-44d6d635c413 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:53.646868 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:53.680044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a9645-6235-4ae3-8468-44d6d635c413 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:53.800122 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a48cb-3f95-4927-a3f2-2474785e4e9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:53.800726 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a48cb-3f95-4927-a3f2-2474785e4e9d service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:53.801003 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a48cb-3f95-4927-a3f2-2474785e4e9d service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:53.801274 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a48cb-3f95-4927-a3f2-2474785e4e9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:53.802886 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-847a48cb-3f95-4927-a3f2-2474785e4e9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:53.880913 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:53.891940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a48cb-3f95-4927-a3f2-2474785e4e9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:53.893525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a48cb-3f95-4927-a3f2-2474785e4e9d service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:53.911226 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 694/1376] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:55.930086 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18da8b-3a54-444a-b49e-aebd02126603 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:55.930684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18da8b-3a54-444a-b49e-aebd02126603 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:55.930852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18da8b-3a54-444a-b49e-aebd02126603 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:55.932227 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 683/1377] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:55.939723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd374e6a-0f49-4748-8f44-ec2b20ee437b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:55.940129 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd374e6a-0f49-4748-8f44-ec2b20ee437b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:55.940280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd374e6a-0f49-4748-8f44-ec2b20ee437b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:56.005258 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-cd374e6a-0f49-4748-8f44-ec2b20ee437b None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:56.006435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cd374e6a-0f49-4748-8f44-ec2b20ee437b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:56.106974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 695/1378] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:56.151647 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18da8b-3a54-444a-b49e-aebd02126603 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:56.153181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4c18da8b-3a54-444a-b49e-aebd02126603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:56.206117 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:56.346437 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18da8b-3a54-444a-b49e-aebd02126603 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:56.467698 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98602728-58fc-428a-8edd-a80855f08555 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:56.468248 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98602728-58fc-428a-8edd-a80855f08555 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:56.468522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98602728-58fc-428a-8edd-a80855f08555 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:56.468801 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98602728-58fc-428a-8edd-a80855f08555 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:56.470398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-98602728-58fc-428a-8edd-a80855f08555 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:56.481943 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:56.568166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98602728-58fc-428a-8edd-a80855f08555 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:56.569767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98602728-58fc-428a-8edd-a80855f08555 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:56.588442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 684/1379] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:58.652398 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83017a6-6951-49a2-97d7-1e9163b0c217 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:58.652943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83017a6-6951-49a2-97d7-1e9163b0c217 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:58.652943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83017a6-6951-49a2-97d7-1e9163b0c217 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:58.654470 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 696/1380] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:18:58.660027 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c71aaa-cb8a-4ac8-8f0b-c1d014914853 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:58.660482 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c71aaa-cb8a-4ac8-8f0b-c1d014914853 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:58.660576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c71aaa-cb8a-4ac8-8f0b-c1d014914853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:58.728302 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b0c71aaa-cb8a-4ac8-8f0b-c1d014914853 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:58.729428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b0c71aaa-cb8a-4ac8-8f0b-c1d014914853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:58.828203 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:58.846497 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 685/1381] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:58.889179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83017a6-6951-49a2-97d7-1e9163b0c217 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:58.890732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c83017a6-6951-49a2-97d7-1e9163b0c217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:58.960017 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:59.061121 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83017a6-6951-49a2-97d7-1e9163b0c217 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:59.194454 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f9030-4ee6-462f-9ec6-9a09d0fe31cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:59.195157 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f9030-4ee6-462f-9ec6-9a09d0fe31cb service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:59.195506 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f9030-4ee6-462f-9ec6-9a09d0fe31cb service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:59.195846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f9030-4ee6-462f-9ec6-9a09d0fe31cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:59.197499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-846f9030-4ee6-462f-9ec6-9a09d0fe31cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:59.222975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:59.298874 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f9030-4ee6-462f-9ec6-9a09d0fe31cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:59.300676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f9030-4ee6-462f-9ec6-9a09d0fe31cb service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:59.320770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 697/1382] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:01.377787 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f590655-971c-42b9-a9b9-a9167e2b6b8c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:01.378353 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f590655-971c-42b9-a9b9-a9167e2b6b8c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:01.378833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f590655-971c-42b9-a9b9-a9167e2b6b8c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:01.380681 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 686/1383] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:19:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:01.386803 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adb505-6865-484c-af16-c2e4086975ca None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:01.387229 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adb505-6865-484c-af16-c2e4086975ca None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:01.387597 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adb505-6865-484c-af16-c2e4086975ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:01.464454 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f3adb505-6865-484c-af16-c2e4086975ca None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:01.465652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f3adb505-6865-484c-af16-c2e4086975ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:01.560110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:01.565126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 698/1384] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:01.615906 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f590655-971c-42b9-a9b9-a9167e2b6b8c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:01.617952 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8f590655-971c-42b9-a9b9-a9167e2b6b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:01.807828 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f590655-971c-42b9-a9b9-a9167e2b6b8c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:01.888124 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:01.940013 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292314af-c4dc-4f45-806d-a0d00a408176 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:01.940632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292314af-c4dc-4f45-806d-a0d00a408176 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:01.940911 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292314af-c4dc-4f45-806d-a0d00a408176 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:01.941185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292314af-c4dc-4f45-806d-a0d00a408176 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:01.942841 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-292314af-c4dc-4f45-806d-a0d00a408176 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:02.040894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292314af-c4dc-4f45-806d-a0d00a408176 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:02.042550 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292314af-c4dc-4f45-806d-a0d00a408176 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:02.062669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 687/1385] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:04.120341 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f5fa0-40f7-4786-b91e-c264910cc6b5 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:04.120875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f5fa0-40f7-4786-b91e-c264910cc6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:04.120875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f5fa0-40f7-4786-b91e-c264910cc6b5 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:04.122452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 699/1386] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:04.127627 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91912b-c4d9-4106-bd91-5aa4ff21fdc4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:04.128044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91912b-c4d9-4106-bd91-5aa4ff21fdc4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:04.128177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91912b-c4d9-4106-bd91-5aa4ff21fdc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:04.200364 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ef91912b-c4d9-4106-bd91-5aa4ff21fdc4 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:04.202077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ef91912b-c4d9-4106-bd91-5aa4ff21fdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:04.228049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:04.310034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 688/1387] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:19:04.354477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f5fa0-40f7-4786-b91e-c264910cc6b5 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:04.356022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-696f5fa0-40f7-4786-b91e-c264910cc6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:04.526563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f5fa0-40f7-4786-b91e-c264910cc6b5 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:04.596838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:04.643369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7c21a-9cd2-4e00-a767-2075f0497eb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:04.644079 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7c21a-9cd2-4e00-a767-2075f0497eb1 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:04.644389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7c21a-9cd2-4e00-a767-2075f0497eb1 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:04.644663 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7c21a-9cd2-4e00-a767-2075f0497eb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:04.646324 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7df7c21a-9cd2-4e00-a767-2075f0497eb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:04.738834 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7c21a-9cd2-4e00-a767-2075f0497eb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:04.740450 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7c21a-9cd2-4e00-a767-2075f0497eb1 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:04.760569 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 700/1388] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:06.782280 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79471516-ca9d-4358-871f-e7b849e37872 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:06.782811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79471516-ca9d-4358-871f-e7b849e37872 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:06.782811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79471516-ca9d-4358-871f-e7b849e37872 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:06.784176 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 689/1389] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:06.789284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79ce30-d6f3-4a95-9a06-93b7e1a81905 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:06.789673 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79ce30-d6f3-4a95-9a06-93b7e1a81905 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:06.789857 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79ce30-d6f3-4a95-9a06-93b7e1a81905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:06.855760 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-2b79ce30-d6f3-4a95-9a06-93b7e1a81905 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:06.856963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2b79ce30-d6f3-4a95-9a06-93b7e1a81905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:06.891641 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:06.955907 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 701/1390] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:07.000749 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79471516-ca9d-4358-871f-e7b849e37872 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:07.002287 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-79471516-ca9d-4358-871f-e7b849e37872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:07.146221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79471516-ca9d-4358-871f-e7b849e37872 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:07.155565 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:07.259251 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfde7db-0cc0-45f7-94e2-c7a33e4b0d51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:07.259963 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfde7db-0cc0-45f7-94e2-c7a33e4b0d51 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:07.260247 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfde7db-0cc0-45f7-94e2-c7a33e4b0d51 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:07.260517 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfde7db-0cc0-45f7-94e2-c7a33e4b0d51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:07.262105 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1cfde7db-0cc0-45f7-94e2-c7a33e4b0d51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:07.351576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfde7db-0cc0-45f7-94e2-c7a33e4b0d51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:07.353189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfde7db-0cc0-45f7-94e2-c7a33e4b0d51 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:07.370882 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 690/1391] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:08.287517 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c37e0-7069-49e1-9955-09619ce57925 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:08.288070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c37e0-7069-49e1-9955-09619ce57925 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:08.288070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c37e0-7069-49e1-9955-09619ce57925 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:08.289399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 702/1392] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:08.294616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb51c7-d28f-493f-a8a8-e130efa988fc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:08.295018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb51c7-d28f-493f-a8a8-e130efa988fc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:08.295711 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb51c7-d28f-493f-a8a8-e130efa988fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:08.311695 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:08.361447 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-86cb51c7-d28f-493f-a8a8-e130efa988fc None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:08.362587 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-86cb51c7-d28f-493f-a8a8-e130efa988fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:08.457016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 691/1393] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:08.514147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b505f1-c8c4-4126-a930-9603c973afb0 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:08.514468 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b505f1-c8c4-4126-a930-9603c973afb0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:08.514750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b505f1-c8c4-4126-a930-9603c973afb0 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:08.516077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 703/1394] 10.222.0.39 () {54 vars in 966 bytes} [Mon Aug 8 02:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:08.521370 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be4af7-ce30-4169-a9e9-1b45adb143be None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:08.521803 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be4af7-ce30-4169-a9e9-1b45adb143be None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:08.522297 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be4af7-ce30-4169-a9e9-1b45adb143be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:08.598483 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-42be4af7-ce30-4169-a9e9-1b45adb143be None None] MFA Rules not processed for user `7776a6a1faf34b369e399d83bb3e18c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:08.599685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-42be4af7-ce30-4169-a9e9-1b45adb143be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:08.635257 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:08.677104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 692/1395] 10.222.0.39 () {58 vars in 1063 bytes} [Mon Aug 8 02:19:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:08.684725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b505f1-c8c4-4126-a930-9603c973afb0 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:08.686516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-03b505f1-c8c4-4126-a930-9603c973afb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:08.788799 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b505f1-c8c4-4126-a930-9603c973afb0 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:08.850096 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:08.863233 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd641d3-84c3-42e0-86d6-b13edbd5bb7c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7776a6a1faf34b369e399d83bb3e18c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:08.863831 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd641d3-84c3-42e0-86d6-b13edbd5bb7c service glance] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:08.864106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd641d3-84c3-42e0-86d6-b13edbd5bb7c service glance] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:08.864376 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd641d3-84c3-42e0-86d6-b13edbd5bb7c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:08.865943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3fd641d3-84c3-42e0-86d6-b13edbd5bb7c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:08.974024 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd641d3-84c3-42e0-86d6-b13edbd5bb7c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:08.975401 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd641d3-84c3-42e0-86d6-b13edbd5bb7c service glance] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:08.993592 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 704/1396] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:09.381000 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038986e6-f184-4f59-9027-40dfb9a66232 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:09.381575 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038986e6-f184-4f59-9027-40dfb9a66232 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:09.381575 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038986e6-f184-4f59-9027-40dfb9a66232 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:09.382908 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 693/1397] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:09.388004 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470947d-847f-480c-8670-7a911de56dcc None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:09.388409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470947d-847f-480c-8670-7a911de56dcc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:09.388551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470947d-847f-480c-8670-7a911de56dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:09.462053 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8470947d-847f-480c-8670-7a911de56dcc None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:09.463283 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8470947d-847f-480c-8670-7a911de56dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:09.506315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:09.565230 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 705/1398] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:09.611664 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038986e6-f184-4f59-9027-40dfb9a66232 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:09.613131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-038986e6-f184-4f59-9027-40dfb9a66232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:09.721878 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adae78-9f83-4fd5-b866-e971cc476d4e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:09.722161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adae78-9f83-4fd5-b866-e971cc476d4e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:09.722472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adae78-9f83-4fd5-b866-e971cc476d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:09.765154 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038986e6-f184-4f59-9027-40dfb9a66232 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:09.788652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f0adae78-9f83-4fd5-b866-e971cc476d4e None None] MFA Rules not processed for user `b88ccd3e5c4c48d3a27f4eaa7874e28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:09.789847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f0adae78-9f83-4fd5-b866-e971cc476d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:09.805791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:09.876294 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 706/1399] 10.222.0.39 () {58 vars in 1009 bytes} [Mon Aug 8 02:19:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:19:09.880162 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6361de5-661e-492e-8e4e-004fafee9bbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:09.880764 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6361de5-661e-492e-8e4e-004fafee9bbf service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:09.881115 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6361de5-661e-492e-8e4e-004fafee9bbf service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:09.881394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6361de5-661e-492e-8e4e-004fafee9bbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:09.883028 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d6361de5-661e-492e-8e4e-004fafee9bbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:09.924509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957b3dc-5804-4741-bde9-5dea619ddc68 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:09.924799 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957b3dc-5804-4741-bde9-5dea619ddc68 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:09.925074 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957b3dc-5804-4741-bde9-5dea619ddc68 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:09.926455 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 707/1400] 10.222.0.39 () {54 vars in 943 bytes} [Mon Aug 8 02:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:19:09.932260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b012fac-8add-41f7-92f2-a8b70bca8fbe None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:09.932543 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b012fac-8add-41f7-92f2-a8b70bca8fbe None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:09.932821 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b012fac-8add-41f7-92f2-a8b70bca8fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:09.976508 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6361de5-661e-492e-8e4e-004fafee9bbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:09.978108 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6361de5-661e-492e-8e4e-004fafee9bbf service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:09.979860 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:09.996886 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 694/1401] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:10.012011 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8b012fac-8add-41f7-92f2-a8b70bca8fbe None None] MFA Rules not processed for user `ac60b4d621234225830c94edfac400e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:10.013385 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8b012fac-8add-41f7-92f2-a8b70bca8fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:10.149824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 708/1402] 10.222.0.39 () {58 vars in 1040 bytes} [Mon Aug 8 02:19:09 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:10.159749 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6361de5-661e-492e-8e4e-004fafee9bbf service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:10.161016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d6361de5-661e-492e-8e4e-004fafee9bbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:10.212028 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:10.255585 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6361de5-661e-492e-8e4e-004fafee9bbf service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:10.359791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec22d46-b2b5-4bac-8fac-0130c728f1f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac60b4d621234225830c94edfac400e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:10.360396 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec22d46-b2b5-4bac-8fac-0130c728f1f0 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:10.360677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec22d46-b2b5-4bac-8fac-0130c728f1f0 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:10.360949 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec22d46-b2b5-4bac-8fac-0130c728f1f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:10.362661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-aec22d46-b2b5-4bac-8fac-0130c728f1f0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:10.449795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:10.454504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec22d46-b2b5-4bac-8fac-0130c728f1f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:10.456516 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec22d46-b2b5-4bac-8fac-0130c728f1f0 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:10.464997 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 695/1403] 10.222.0.39 () {60 vars in 1384 bytes} [Mon Aug 8 02:19:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:19:12.043647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee9b00-d406-4147-aafb-0bb61aee71ce None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:12.044349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee9b00-d406-4147-aafb-0bb61aee71ce None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:12.044349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee9b00-d406-4147-aafb-0bb61aee71ce None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:12.046374 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 709/1404] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:19:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:12.051327 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02c8bd-6ec7-442d-a484-5c0a8119a5d0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:12.053980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02c8bd-6ec7-442d-a484-5c0a8119a5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:12.054456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02c8bd-6ec7-442d-a484-5c0a8119a5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:12.138224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ce02c8bd-6ec7-442d-a484-5c0a8119a5d0 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:12.139528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ce02c8bd-6ec7-442d-a484-5c0a8119a5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:12.255309 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 696/1405] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:12.292098 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee9b00-d406-4147-aafb-0bb61aee71ce None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:12.293769 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-89ee9b00-d406-4147-aafb-0bb61aee71ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:12.398944 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:12.466205 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee9b00-d406-4147-aafb-0bb61aee71ce None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:12.596010 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5777f-a975-43ea-85e9-0bf8922822b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:12.596531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5777f-a975-43ea-85e9-0bf8922822b4 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:12.596807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5777f-a975-43ea-85e9-0bf8922822b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:12.597079 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5777f-a975-43ea-85e9-0bf8922822b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:12.598773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-85a5777f-a975-43ea-85e9-0bf8922822b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:12.661423 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:12.697830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5777f-a975-43ea-85e9-0bf8922822b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:12.699508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5777f-a975-43ea-85e9-0bf8922822b4 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:12.719441 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 710/1406] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:14.769520 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9268c4-4179-4759-905c-e002b79730cd None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:14.770651 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9268c4-4179-4759-905c-e002b79730cd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:14.771128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9268c4-4179-4759-905c-e002b79730cd None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:14.773078 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 697/1407] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:19:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:14.779459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694aec6-fde3-48ab-8542-df3d81822995 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:14.780478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694aec6-fde3-48ab-8542-df3d81822995 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:14.780984 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694aec6-fde3-48ab-8542-df3d81822995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:14.856019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b694aec6-fde3-48ab-8542-df3d81822995 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:14.857259 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b694aec6-fde3-48ab-8542-df3d81822995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:14.963361 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 711/1408] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:15.008952 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9268c4-4179-4759-905c-e002b79730cd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:15.009592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8c9268c4-4179-4759-905c-e002b79730cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:15.093761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:15.188938 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9268c4-4179-4759-905c-e002b79730cd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:15.312927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c1431-f85b-4f7f-91f5-a0749d13297c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:15.313536 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c1431-f85b-4f7f-91f5-a0749d13297c service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:15.313818 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c1431-f85b-4f7f-91f5-a0749d13297c service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:15.314082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c1431-f85b-4f7f-91f5-a0749d13297c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:15.315927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fb6c1431-f85b-4f7f-91f5-a0749d13297c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:15.359964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:15.428008 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c1431-f85b-4f7f-91f5-a0749d13297c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:15.429646 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c1431-f85b-4f7f-91f5-a0749d13297c service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:15.456752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 698/1409] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:24.961151 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=85 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:25.450582 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=30 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:32.690721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50552605-ebbe-4cc9-ae02-2906fa7b7652 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:32.691282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50552605-ebbe-4cc9-ae02-2906fa7b7652 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:32.691282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50552605-ebbe-4cc9-ae02-2906fa7b7652 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:32.692641 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 712/1410] 10.222.0.39 () {54 vars in 918 bytes} [Mon Aug 8 02:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:32.697783 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50882d-4475-446e-a4e0-616ec227cb75 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:32.698194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50882d-4475-446e-a4e0-616ec227cb75 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:32.698321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50882d-4475-446e-a4e0-616ec227cb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:32.779201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8b50882d-4475-446e-a4e0-616ec227cb75 None None] MFA Rules not processed for user `f62745b92a864480a9438de581e1d05e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:32.780437 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8b50882d-4475-446e-a4e0-616ec227cb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:32.884724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 699/1411] 10.222.0.39 () {58 vars in 1015 bytes} [Mon Aug 8 02:19:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:32.898789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fbf23-78cd-4880-97ae-4fd2775ab27e None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:32.899099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fbf23-78cd-4880-97ae-4fd2775ab27e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:32.899374 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fbf23-78cd-4880-97ae-4fd2775ab27e None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:32.900685 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 713/1412] 10.222.0.39 () {54 vars in 918 bytes} [Mon Aug 8 02:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:32.905401 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea350a-6b7a-4faf-86d8-b5538be205e0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:32.905736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea350a-6b7a-4faf-86d8-b5538be205e0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:32.906005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea350a-6b7a-4faf-86d8-b5538be205e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:32.976413 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:32.979548 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cfea350a-6b7a-4faf-86d8-b5538be205e0 None None] MFA Rules not processed for user `c4238566a9c248ffbfa314f47f02670c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:32.981192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cfea350a-6b7a-4faf-86d8-b5538be205e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:33.096253 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 700/1413] 10.222.0.39 () {58 vars in 1015 bytes} [Mon Aug 8 02:19:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:19:35.555838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744b1c0-15fc-49b7-973b-76d185dc8def None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:35.556399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744b1c0-15fc-49b7-973b-76d185dc8def None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:35.556399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744b1c0-15fc-49b7-973b-76d185dc8def None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:35.557701 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:19:35.563227 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111161d-dbab-4b1d-8eb1-3df92823bc02 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:35.563673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111161d-dbab-4b1d-8eb1-3df92823bc02 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:35.563782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111161d-dbab-4b1d-8eb1-3df92823bc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:35.627371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5111161d-dbab-4b1d-8eb1-3df92823bc02 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:35.628474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5111161d-dbab-4b1d-8eb1-3df92823bc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:35.682214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:35.741054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:35.784593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744b1c0-15fc-49b7-973b-76d185dc8def None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:35.786106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f744b1c0-15fc-49b7-973b-76d185dc8def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:35.928520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744b1c0-15fc-49b7-973b-76d185dc8def None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:35.999808 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:36.039647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41acb8f-8111-4c56-b9ae-2380a2c0c69f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:36.040254 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41acb8f-8111-4c56-b9ae-2380a2c0c69f service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:36.040528 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41acb8f-8111-4c56-b9ae-2380a2c0c69f service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:36.040795 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41acb8f-8111-4c56-b9ae-2380a2c0c69f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:36.042414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b41acb8f-8111-4c56-b9ae-2380a2c0c69f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:36.131507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41acb8f-8111-4c56-b9ae-2380a2c0c69f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:36.133088 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41acb8f-8111-4c56-b9ae-2380a2c0c69f service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:36.150906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 715/1416] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:37.047539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f688c8-7449-4b5c-96de-0837c20bdd53 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:37.048123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f688c8-7449-4b5c-96de-0837c20bdd53 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:37.048123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f688c8-7449-4b5c-96de-0837c20bdd53 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:37.049438 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:37.054353 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a7b64-1b0a-43f3-a3ae-f0a33908bcd8 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:37.054776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a7b64-1b0a-43f3-a3ae-f0a33908bcd8 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:37.054900 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a7b64-1b0a-43f3-a3ae-f0a33908bcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:37.132209 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-759a7b64-1b0a-43f3-a3ae-f0a33908bcd8 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:37.133425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-759a7b64-1b0a-43f3-a3ae-f0a33908bcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:37.160353 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:37.229757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:37.380066 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f688c8-7449-4b5c-96de-0837c20bdd53 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:37.381506 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-08f688c8-7449-4b5c-96de-0837c20bdd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:37.525240 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f688c8-7449-4b5c-96de-0837c20bdd53 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:37.565592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:37.638177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1988334-82f8-4e47-87fa-5c84df195317 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:37.638736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1988334-82f8-4e47-87fa-5c84df195317 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:37.639014 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1988334-82f8-4e47-87fa-5c84df195317 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:37.639286 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1988334-82f8-4e47-87fa-5c84df195317 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:37.640882 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d1988334-82f8-4e47-87fa-5c84df195317 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:37.729556 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1988334-82f8-4e47-87fa-5c84df195317 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:37.731198 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1988334-82f8-4e47-87fa-5c84df195317 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:37.749602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 703/1419] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:38.627594 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9475673-a773-4da0-9750-d6377b3ebaa8 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:38.628152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9475673-a773-4da0-9750-d6377b3ebaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:38.628258 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9475673-a773-4da0-9750-d6377b3ebaa8 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:38.629523 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:38.634343 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6950cb0-7932-42c2-a2e8-361e0e85e1e0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:38.634719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6950cb0-7932-42c2-a2e8-361e0e85e1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:38.634901 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6950cb0-7932-42c2-a2e8-361e0e85e1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:38.688726 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:38.711169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d6950cb0-7932-42c2-a2e8-361e0e85e1e0 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:38.712317 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d6950cb0-7932-42c2-a2e8-361e0e85e1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:38.808353 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:38.961462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9475673-a773-4da0-9750-d6377b3ebaa8 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:38.962951 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b9475673-a773-4da0-9750-d6377b3ebaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:39.103725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9475673-a773-4da0-9750-d6377b3ebaa8 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:39.116473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:39.214862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2128d04-68f7-4760-97c8-3172ba007778 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:39.215390 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2128d04-68f7-4760-97c8-3172ba007778 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:39.215666 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2128d04-68f7-4760-97c8-3172ba007778 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:39.215940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2128d04-68f7-4760-97c8-3172ba007778 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:39.218026 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c2128d04-68f7-4760-97c8-3172ba007778 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:39.305174 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2128d04-68f7-4760-97c8-3172ba007778 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:39.306794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2128d04-68f7-4760-97c8-3172ba007778 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:39.324438 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 718/1422] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:40.179707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559d23d-2405-4b0f-91e7-5e76b27a310e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:40.180255 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559d23d-2405-4b0f-91e7-5e76b27a310e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:40.180255 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559d23d-2405-4b0f-91e7-5e76b27a310e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:40.181613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:40.187204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a992cdc-e055-42c9-b5cc-268b0fe4a023 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:40.187628 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a992cdc-e055-42c9-b5cc-268b0fe4a023 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:40.187747 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a992cdc-e055-42c9-b5cc-268b0fe4a023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:40.207588 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:40.264455 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7a992cdc-e055-42c9-b5cc-268b0fe4a023 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:40.265676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7a992cdc-e055-42c9-b5cc-268b0fe4a023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:40.480105 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:40.629066 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559d23d-2405-4b0f-91e7-5e76b27a310e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:40.630614 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e559d23d-2405-4b0f-91e7-5e76b27a310e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:40.739049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:40.774305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559d23d-2405-4b0f-91e7-5e76b27a310e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:40.890294 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a4459-0dcb-428b-863a-2f1a273c9892 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:40.890920 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a4459-0dcb-428b-863a-2f1a273c9892 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:40.891204 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a4459-0dcb-428b-863a-2f1a273c9892 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:40.891474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a4459-0dcb-428b-863a-2f1a273c9892 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:40.893065 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-102a4459-0dcb-428b-863a-2f1a273c9892 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:40.968264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:40.982378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a4459-0dcb-428b-863a-2f1a273c9892 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:40.983971 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a4459-0dcb-428b-863a-2f1a273c9892 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:41.001730 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 706/1425] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:41.860851 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40033d-ffc3-4569-a22d-3f1341cb53c5 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:41.861395 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40033d-ffc3-4569-a22d-3f1341cb53c5 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:41.861395 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40033d-ffc3-4569-a22d-3f1341cb53c5 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:41.862745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:41.867670 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45510b00-2ecf-4a5b-825b-40d60907a2d8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:41.868118 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45510b00-2ecf-4a5b-825b-40d60907a2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:41.868217 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45510b00-2ecf-4a5b-825b-40d60907a2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:41.942448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-45510b00-2ecf-4a5b-825b-40d60907a2d8 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:41.943646 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-45510b00-2ecf-4a5b-825b-40d60907a2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:42.037257 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:42.187911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40033d-ffc3-4569-a22d-3f1341cb53c5 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:42.189363 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0b40033d-ffc3-4569-a22d-3f1341cb53c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:42.270072 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:42.330957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40033d-ffc3-4569-a22d-3f1341cb53c5 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:42.449490 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0f276-47d6-45c8-bae4-f054bc7e619e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:42.450088 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0f276-47d6-45c8-bae4-f054bc7e619e service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:42.450393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0f276-47d6-45c8-bae4-f054bc7e619e service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:42.450660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0f276-47d6-45c8-bae4-f054bc7e619e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:42.452743 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a6d0f276-47d6-45c8-bae4-f054bc7e619e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:42.499269 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:42.540226 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d0f276-47d6-45c8-bae4-f054bc7e619e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:42.541808 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d0f276-47d6-45c8-bae4-f054bc7e619e service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:42.559257 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 721/1428] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:43.382060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d43801-4b95-47bf-b33c-49f184e8f7c2 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:43.382649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d43801-4b95-47bf-b33c-49f184e8f7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:43.382649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d43801-4b95-47bf-b33c-49f184e8f7c2 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:43.383988 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:43.389040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15ed40-25c0-4279-93ac-c0461db15068 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:43.389466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15ed40-25c0-4279-93ac-c0461db15068 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:43.389560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15ed40-25c0-4279-93ac-c0461db15068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:43.465821 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-5c15ed40-25c0-4279-93ac-c0461db15068 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:43.467048 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5c15ed40-25c0-4279-93ac-c0461db15068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:43.561963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:43.710636 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d43801-4b95-47bf-b33c-49f184e8f7c2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:43.712048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-61d43801-4b95-47bf-b33c-49f184e8f7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:43.759590 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:43.856003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d43801-4b95-47bf-b33c-49f184e8f7c2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:43.965205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd75cba-ed74-4589-8951-fbdf4c904f17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:43.965733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd75cba-ed74-4589-8951-fbdf4c904f17 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:43.966008 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd75cba-ed74-4589-8951-fbdf4c904f17 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:43.966300 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd75cba-ed74-4589-8951-fbdf4c904f17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:43.968483 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7cd75cba-ed74-4589-8951-fbdf4c904f17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:43.975213 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:44.055452 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd75cba-ed74-4589-8951-fbdf4c904f17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:44.057060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd75cba-ed74-4589-8951-fbdf4c904f17 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:44.075015 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 709/1431] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:44.871527 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141dc1b-2c7e-41da-9cef-0a27db343f6f None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:44.872071 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141dc1b-2c7e-41da-9cef-0a27db343f6f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:44.872191 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141dc1b-2c7e-41da-9cef-0a27db343f6f None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:44.874119 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:19:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:44.878960 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bc8b2-da1e-4120-b1dc-238270f985f9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:44.879408 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bc8b2-da1e-4120-b1dc-238270f985f9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:44.879506 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bc8b2-da1e-4120-b1dc-238270f985f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:44.955451 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-167bc8b2-da1e-4120-b1dc-238270f985f9 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:44.956647 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-167bc8b2-da1e-4120-b1dc-238270f985f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:45.022964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:45.052271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:45.202153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141dc1b-2c7e-41da-9cef-0a27db343f6f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:45.203608 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f141dc1b-2c7e-41da-9cef-0a27db343f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:45.220975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:45.345091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141dc1b-2c7e-41da-9cef-0a27db343f6f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:45.429154 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:45.456654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9b94e-3bbf-4e26-a9de-d99138c19081 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:45.457196 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9b94e-3bbf-4e26-a9de-d99138c19081 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:45.457473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9b94e-3bbf-4e26-a9de-d99138c19081 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:45.457741 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9b94e-3bbf-4e26-a9de-d99138c19081 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:45.459383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7bc9b94e-3bbf-4e26-a9de-d99138c19081 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:45.546764 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9b94e-3bbf-4e26-a9de-d99138c19081 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:45.548338 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9b94e-3bbf-4e26-a9de-d99138c19081 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:45.566116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 724/1434] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:46.362017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fced28-7436-4f1a-96da-972a088d8836 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:46.362600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fced28-7436-4f1a-96da-972a088d8836 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:46.362600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fced28-7436-4f1a-96da-972a088d8836 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:46.363858 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:46.368668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc96c2-3574-4b85-b92b-eec0708f6596 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:46.369070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc96c2-3574-4b85-b92b-eec0708f6596 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:46.369218 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc96c2-3574-4b85-b92b-eec0708f6596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:46.445649 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-64cc96c2-3574-4b85-b92b-eec0708f6596 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:46.446897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-64cc96c2-3574-4b85-b92b-eec0708f6596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:46.488025 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:46.541936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:46.691007 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fced28-7436-4f1a-96da-972a088d8836 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:46.692501 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a7fced28-7436-4f1a-96da-972a088d8836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:46.834884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fced28-7436-4f1a-96da-972a088d8836 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:46.887795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:46.946710 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971e733-17ac-42c4-9064-5d423bfc8f4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:46.947230 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971e733-17ac-42c4-9064-5d423bfc8f4b service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:46.947506 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971e733-17ac-42c4-9064-5d423bfc8f4b service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:46.947773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971e733-17ac-42c4-9064-5d423bfc8f4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:46.949347 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0971e733-17ac-42c4-9064-5d423bfc8f4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:47.036832 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971e733-17ac-42c4-9064-5d423bfc8f4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:47.038453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971e733-17ac-42c4-9064-5d423bfc8f4b service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:47.056099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 712/1437] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:47.860561 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22cee6-2c3c-459a-8d22-54cc74122994 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:47.861132 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22cee6-2c3c-459a-8d22-54cc74122994 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:47.861132 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22cee6-2c3c-459a-8d22-54cc74122994 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:47.862444 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:47.867295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa7a31-5a46-49ea-aa10-181768a4eb17 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:47.867726 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa7a31-5a46-49ea-aa10-181768a4eb17 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:47.867875 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa7a31-5a46-49ea-aa10-181768a4eb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:47.943837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bbfa7a31-5a46-49ea-aa10-181768a4eb17 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:47.945051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bbfa7a31-5a46-49ea-aa10-181768a4eb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:47.947072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:48.167097 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:48.316610 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22cee6-2c3c-459a-8d22-54cc74122994 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:48.318112 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0d22cee6-2c3c-459a-8d22-54cc74122994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:48.466091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22cee6-2c3c-459a-8d22-54cc74122994 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:48.493869 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:48.579955 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836e93e-3193-4531-8e58-3a7909a9509f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:48.580555 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836e93e-3193-4531-8e58-3a7909a9509f service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:48.580830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836e93e-3193-4531-8e58-3a7909a9509f service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:48.581107 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836e93e-3193-4531-8e58-3a7909a9509f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:48.582730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c836e93e-3193-4531-8e58-3a7909a9509f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:48.669859 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836e93e-3193-4531-8e58-3a7909a9509f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:48.671474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836e93e-3193-4531-8e58-3a7909a9509f service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:48.689374 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 727/1440] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:50.590860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff09905-74d1-4bfe-9986-b46502962e3b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:50.591374 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff09905-74d1-4bfe-9986-b46502962e3b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:50.591511 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff09905-74d1-4bfe-9986-b46502962e3b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:50.592891 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:50.598126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61bf4e-627a-41d0-873c-48640b084b61 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:50.598590 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61bf4e-627a-41d0-873c-48640b084b61 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:50.598718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61bf4e-627a-41d0-873c-48640b084b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:50.639186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:50.664719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6d61bf4e-627a-41d0-873c-48640b084b61 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:50.665901 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6d61bf4e-627a-41d0-873c-48640b084b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:50.760472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:50.799573 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff09905-74d1-4bfe-9986-b46502962e3b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:50.801084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2ff09905-74d1-4bfe-9986-b46502962e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:50.901486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff09905-74d1-4bfe-9986-b46502962e3b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:50.932081 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:50.973037 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05492004-70a5-4ded-9d17-22c42457a03a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:50.973624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05492004-70a5-4ded-9d17-22c42457a03a service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:50.973896 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05492004-70a5-4ded-9d17-22c42457a03a service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:50.974160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05492004-70a5-4ded-9d17-22c42457a03a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:50.975735 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-05492004-70a5-4ded-9d17-22c42457a03a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:51.081955 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05492004-70a5-4ded-9d17-22c42457a03a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:51.083458 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05492004-70a5-4ded-9d17-22c42457a03a service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:51.101481 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 715/1443] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:53.239576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49562ff-429c-470f-8a93-1c062f9aa775 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:53.240118 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49562ff-429c-470f-8a93-1c062f9aa775 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:53.240118 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49562ff-429c-470f-8a93-1c062f9aa775 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:53.241451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:19:53.246782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8309d-6a93-4fff-a689-87f1fb03d0fb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:53.247182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8309d-6a93-4fff-a689-87f1fb03d0fb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:53.247319 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8309d-6a93-4fff-a689-87f1fb03d0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:53.312200 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-caa8309d-6a93-4fff-a689-87f1fb03d0fb None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:53.313394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-caa8309d-6a93-4fff-a689-87f1fb03d0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:53.325486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:53.407290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:53.446744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49562ff-429c-470f-8a93-1c062f9aa775 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:53.448367 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b49562ff-429c-470f-8a93-1c062f9aa775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:53.549270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49562ff-429c-470f-8a93-1c062f9aa775 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:53.561653 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:53.622469 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663853c9-1001-4205-be6e-669b09f56f55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:53.623078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663853c9-1001-4205-be6e-669b09f56f55 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:53.623354 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663853c9-1001-4205-be6e-669b09f56f55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:53.623624 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663853c9-1001-4205-be6e-669b09f56f55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:53.625222 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-663853c9-1001-4205-be6e-669b09f56f55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:53.730720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663853c9-1001-4205-be6e-669b09f56f55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:53.732159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663853c9-1001-4205-be6e-669b09f56f55 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:53.749503 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 730/1446] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:56.019669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1550f8-2d86-49d2-8197-b38e8b122fdb None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:56.020233 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1550f8-2d86-49d2-8197-b38e8b122fdb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:56.020233 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1550f8-2d86-49d2-8197-b38e8b122fdb None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:56.021520 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:56.026686 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f04385-3568-4564-a837-e2de0d492e78 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:56.027132 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f04385-3568-4564-a837-e2de0d492e78 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:56.027229 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f04385-3568-4564-a837-e2de0d492e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:56.074649 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:56.092094 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-94f04385-3568-4564-a837-e2de0d492e78 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:56.093299 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-94f04385-3568-4564-a837-e2de0d492e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:56.193718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:56.232885 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1550f8-2d86-49d2-8197-b38e8b122fdb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:56.234452 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7e1550f8-2d86-49d2-8197-b38e8b122fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:56.337846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1550f8-2d86-49d2-8197-b38e8b122fdb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:56.377669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:56.410976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248efc22-18c2-4efa-9192-c431ed06959e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:56.411606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248efc22-18c2-4efa-9192-c431ed06959e service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:56.411880 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248efc22-18c2-4efa-9192-c431ed06959e service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:56.412145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248efc22-18c2-4efa-9192-c431ed06959e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:56.413735 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-248efc22-18c2-4efa-9192-c431ed06959e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:56.522166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248efc22-18c2-4efa-9192-c431ed06959e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:56.523624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248efc22-18c2-4efa-9192-c431ed06959e service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:56.541858 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 718/1449] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:58.658159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e547cd0-9f15-4823-9299-9e9308278999 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:58.658739 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e547cd0-9f15-4823-9299-9e9308278999 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:58.658739 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e547cd0-9f15-4823-9299-9e9308278999 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:58.660224 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:58.665548 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b57b5-853a-45bd-8398-91d5dae42d6b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:58.665988 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b57b5-853a-45bd-8398-91d5dae42d6b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:58.666095 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b57b5-853a-45bd-8398-91d5dae42d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:58.730884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-194b57b5-853a-45bd-8398-91d5dae42d6b None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:58.732057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-194b57b5-853a-45bd-8398-91d5dae42d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:58.752614 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:58.829491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:58.867937 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e547cd0-9f15-4823-9299-9e9308278999 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:58.869449 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8e547cd0-9f15-4823-9299-9e9308278999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:58.970933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e547cd0-9f15-4823-9299-9e9308278999 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:58.990498 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:59.042385 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c694f-2224-4749-b3eb-d5d80153b227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:59.042974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c694f-2224-4749-b3eb-d5d80153b227 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:59.043249 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c694f-2224-4749-b3eb-d5d80153b227 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:59.043515 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c694f-2224-4749-b3eb-d5d80153b227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:59.045592 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-315c694f-2224-4749-b3eb-d5d80153b227 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:59.153485 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c694f-2224-4749-b3eb-d5d80153b227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:59.154963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c694f-2224-4749-b3eb-d5d80153b227 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:59.173099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 733/1452] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:20:01.283305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402cb16-9b99-4a0a-ae7c-3ae1ae2f0d6a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:01.283850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402cb16-9b99-4a0a-ae7c-3ae1ae2f0d6a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:01.283850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402cb16-9b99-4a0a-ae7c-3ae1ae2f0d6a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:01.285215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:01.290343 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184e053-06cb-4eaa-9509-1a64f9f953a6 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:01.290795 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184e053-06cb-4eaa-9509-1a64f9f953a6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:01.290909 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184e053-06cb-4eaa-9509-1a64f9f953a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:01.353609 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:01.356160 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-4184e053-06cb-4eaa-9509-1a64f9f953a6 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:01.358417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4184e053-06cb-4eaa-9509-1a64f9f953a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:01.455229 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:20:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:01.493527 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402cb16-9b99-4a0a-ae7c-3ae1ae2f0d6a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:01.495136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6402cb16-9b99-4a0a-ae7c-3ae1ae2f0d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:01.595611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402cb16-9b99-4a0a-ae7c-3ae1ae2f0d6a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:01.642474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:01.666736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a3eee-757d-4225-927b-cbff3b4f6d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:01.667327 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a3eee-757d-4225-927b-cbff3b4f6d75 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:01.667602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a3eee-757d-4225-927b-cbff3b4f6d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:01.667867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a3eee-757d-4225-927b-cbff3b4f6d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:01.670032 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a04a3eee-757d-4225-927b-cbff3b4f6d75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:01.775576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a3eee-757d-4225-927b-cbff3b4f6d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:01.777001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a3eee-757d-4225-927b-cbff3b4f6d75 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:01.795686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 721/1455] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:20:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:20:03.867398 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c565b-79b5-485f-a3ef-bb1232da5615 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:03.867965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c565b-79b5-485f-a3ef-bb1232da5615 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:03.867965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c565b-79b5-485f-a3ef-bb1232da5615 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:03.869292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:03.875280 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b74096-c387-4da9-9e8d-cf5f8eb62f4e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:03.875708 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b74096-c387-4da9-9e8d-cf5f8eb62f4e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:03.875826 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b74096-c387-4da9-9e8d-cf5f8eb62f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:03.941557 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-22b74096-c387-4da9-9e8d-cf5f8eb62f4e None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:03.942806 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-22b74096-c387-4da9-9e8d-cf5f8eb62f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:03.972095 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:04.038113 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:04.076566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c565b-79b5-485f-a3ef-bb1232da5615 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:04.078104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d76c565b-79b5-485f-a3ef-bb1232da5615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:04.180511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c565b-79b5-485f-a3ef-bb1232da5615 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:04.208995 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:04.253477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ea973-88ee-469a-9cc1-b041f2b71e98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:04.254069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ea973-88ee-469a-9cc1-b041f2b71e98 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:04.254373 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ea973-88ee-469a-9cc1-b041f2b71e98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:04.254642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ea973-88ee-469a-9cc1-b041f2b71e98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:04.256210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6c8ea973-88ee-469a-9cc1-b041f2b71e98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:04.361302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ea973-88ee-469a-9cc1-b041f2b71e98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:04.362750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ea973-88ee-469a-9cc1-b041f2b71e98 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:04.380828 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 736/1458] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:20:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:20:06.432873 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388ae7b-2919-4a1d-8bcd-18418f5f23ae None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:06.433394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388ae7b-2919-4a1d-8bcd-18418f5f23ae None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:06.433394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388ae7b-2919-4a1d-8bcd-18418f5f23ae None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:06.434730 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:06.440125 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56940092-dcd2-428b-aa4a-6794e985e4db None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:06.440556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56940092-dcd2-428b-aa4a-6794e985e4db None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:06.440682 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56940092-dcd2-428b-aa4a-6794e985e4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:06.505538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-56940092-dcd2-428b-aa4a-6794e985e4db None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:06.506779 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-56940092-dcd2-428b-aa4a-6794e985e4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:06.515437 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:06.603319 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:06.640559 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388ae7b-2919-4a1d-8bcd-18418f5f23ae None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:06.642061 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2388ae7b-2919-4a1d-8bcd-18418f5f23ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:06.747607 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388ae7b-2919-4a1d-8bcd-18418f5f23ae None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:06.800764 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:06.818922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20008d1-b223-49a0-b5b7-5126fd660a30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:06.819518 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20008d1-b223-49a0-b5b7-5126fd660a30 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:06.819793 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20008d1-b223-49a0-b5b7-5126fd660a30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:06.820061 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20008d1-b223-49a0-b5b7-5126fd660a30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:06.821634 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d20008d1-b223-49a0-b5b7-5126fd660a30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:06.927641 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20008d1-b223-49a0-b5b7-5126fd660a30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:06.929094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20008d1-b223-49a0-b5b7-5126fd660a30 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:06.947489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 724/1461] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:20:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:20:09.003007 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df5935-72b0-4af8-9b15-0a52bc2f4b1e None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:09.003582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df5935-72b0-4af8-9b15-0a52bc2f4b1e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:09.003582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df5935-72b0-4af8-9b15-0a52bc2f4b1e None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:09.004887 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:09.010149 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5535e28-52c6-44eb-b1bb-32d47da0d363 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:09.010559 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5535e28-52c6-44eb-b1bb-32d47da0d363 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:09.010749 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5535e28-52c6-44eb-b1bb-32d47da0d363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:09.074699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c5535e28-52c6-44eb-b1bb-32d47da0d363 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:09.075880 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c5535e28-52c6-44eb-b1bb-32d47da0d363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:09.113579 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:09.171298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:09.208160 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df5935-72b0-4af8-9b15-0a52bc2f4b1e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:09.209687 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-16df5935-72b0-4af8-9b15-0a52bc2f4b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:09.310681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df5935-72b0-4af8-9b15-0a52bc2f4b1e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:09.347365 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:09.382308 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1dbf4a-0d78-4458-9f57-54b5955c91e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:09.382904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dbf4a-0d78-4458-9f57-54b5955c91e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:09.383182 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dbf4a-0d78-4458-9f57-54b5955c91e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:09.383453 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dbf4a-0d78-4458-9f57-54b5955c91e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:09.385026 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0c1dbf4a-0d78-4458-9f57-54b5955c91e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:09.500581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1dbf4a-0d78-4458-9f57-54b5955c91e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:09.502006 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1dbf4a-0d78-4458-9f57-54b5955c91e3 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:09.520115 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 739/1464] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:20:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:20:11.538845 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd26937-4e51-41a4-93cf-697d27536250 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:11.539366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd26937-4e51-41a4-93cf-697d27536250 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:11.539464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd26937-4e51-41a4-93cf-697d27536250 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:11.540795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:11.546102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d11a9-a8c4-4c14-ab3c-072f49ef4c0d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:11.546538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d11a9-a8c4-4c14-ab3c-072f49ef4c0d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:11.546801 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d11a9-a8c4-4c14-ab3c-072f49ef4c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:11.612118 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-502d11a9-a8c4-4c14-ab3c-072f49ef4c0d None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:11.613330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-502d11a9-a8c4-4c14-ab3c-072f49ef4c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:11.631518 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:11.709952 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:11.751844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd26937-4e51-41a4-93cf-697d27536250 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:11.753388 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-abd26937-4e51-41a4-93cf-697d27536250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:11.855384 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd26937-4e51-41a4-93cf-697d27536250 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:11.920265 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:11.927247 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fec63e-4f4b-41a4-897d-3d56cb97fa85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:11.927834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fec63e-4f4b-41a4-897d-3d56cb97fa85 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:11.928106 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fec63e-4f4b-41a4-897d-3d56cb97fa85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:11.928371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fec63e-4f4b-41a4-897d-3d56cb97fa85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:11.929934 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-13fec63e-4f4b-41a4-897d-3d56cb97fa85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:12.040782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fec63e-4f4b-41a4-897d-3d56cb97fa85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:12.042216 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fec63e-4f4b-41a4-897d-3d56cb97fa85 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:12.061086 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 727/1467] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:20:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:20:14.052410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda1c52-fa8b-4c67-88fc-368f3fc4f15b None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:14.052988 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda1c52-fa8b-4c67-88fc-368f3fc4f15b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:14.052988 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda1c52-fa8b-4c67-88fc-368f3fc4f15b None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:14.054301 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:14.059620 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d56d0-268b-47a5-80e2-438a055c1a74 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:14.060060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d56d0-268b-47a5-80e2-438a055c1a74 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:14.060154 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d56d0-268b-47a5-80e2-438a055c1a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:14.126503 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7d9d56d0-268b-47a5-80e2-438a055c1a74 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:14.127714 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7d9d56d0-268b-47a5-80e2-438a055c1a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:14.175204 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:14.223445 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:14.259426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda1c52-fa8b-4c67-88fc-368f3fc4f15b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:14.260949 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3bda1c52-fa8b-4c67-88fc-368f3fc4f15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:14.364661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda1c52-fa8b-4c67-88fc-368f3fc4f15b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:14.410589 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:14.436850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6977c-964b-457c-a37d-48632d7d964d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:14.437445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6977c-964b-457c-a37d-48632d7d964d service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:14.437719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6977c-964b-457c-a37d-48632d7d964d service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:14.437985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6977c-964b-457c-a37d-48632d7d964d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:14.439594 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-57a6977c-964b-457c-a37d-48632d7d964d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:14.548279 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6977c-964b-457c-a37d-48632d7d964d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:14.549701 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6977c-964b-457c-a37d-48632d7d964d service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:14.567939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 742/1470] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:20:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:20:24.221613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=22 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:24.565737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=66 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:33.950262 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415e0be-ce5f-4bf3-b843-0ece13a5627a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:33.950818 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415e0be-ce5f-4bf3-b843-0ece13a5627a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:33.950818 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415e0be-ce5f-4bf3-b843-0ece13a5627a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:33.952148 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 729/1471] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:33.957382 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f44d66-db98-4805-b2a8-e303ecad65c3 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:33.957798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f44d66-db98-4805-b2a8-e303ecad65c3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:33.957904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f44d66-db98-4805-b2a8-e303ecad65c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:34.023069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-66f44d66-db98-4805-b2a8-e303ecad65c3 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:34.024277 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-66f44d66-db98-4805-b2a8-e303ecad65c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:34.128696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 743/1472] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:34.213072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bd4af-bfcc-454c-8106-250ba8c3c867 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:34.213366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bd4af-bfcc-454c-8106-250ba8c3c867 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:34.213637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bd4af-bfcc-454c-8106-250ba8c3c867 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:34.214999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 730/1473] 10.222.0.39 () {54 vars in 966 bytes} [Mon Aug 8 02:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:34.220545 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019ac0c-8212-4d24-9fc0-035ff83a6376 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:34.220865 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019ac0c-8212-4d24-9fc0-035ff83a6376 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:34.221144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019ac0c-8212-4d24-9fc0-035ff83a6376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:34.296748 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f019ac0c-8212-4d24-9fc0-035ff83a6376 None None] MFA Rules not processed for user `7776a6a1faf34b369e399d83bb3e18c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:34.297960 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f019ac0c-8212-4d24-9fc0-035ff83a6376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:34.319015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:34.374650 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 744/1474] 10.222.0.39 () {58 vars in 1063 bytes} [Mon Aug 8 02:20:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:34.382211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bd4af-bfcc-454c-8106-250ba8c3c867 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:34.383726 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6b9bd4af-bfcc-454c-8106-250ba8c3c867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:34.505850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bd4af-bfcc-454c-8106-250ba8c3c867 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:34.577457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92caf686-14f3-447b-9313-950be648f940 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7776a6a1faf34b369e399d83bb3e18c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:34.578050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92caf686-14f3-447b-9313-950be648f940 service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:34.578358 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92caf686-14f3-447b-9313-950be648f940 service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:34.578622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92caf686-14f3-447b-9313-950be648f940 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:34.580197 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-92caf686-14f3-447b-9313-950be648f940 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:34.634407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:34.690890 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92caf686-14f3-447b-9313-950be648f940 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:34.692431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92caf686-14f3-447b-9313-950be648f940 service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:34.711478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 731/1475] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:20:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:20:36.785804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463b6c7-1646-4116-81c1-7021a4f9e393 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:36.786383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463b6c7-1646-4116-81c1-7021a4f9e393 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:36.786383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463b6c7-1646-4116-81c1-7021a4f9e393 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:36.787731 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 745/1476] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:36.792855 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989af5e-4cea-4ce3-a7dc-bdf45cdc63d6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:36.793304 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989af5e-4cea-4ce3-a7dc-bdf45cdc63d6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:36.793415 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989af5e-4cea-4ce3-a7dc-bdf45cdc63d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:36.859048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c989af5e-4cea-4ce3-a7dc-bdf45cdc63d6 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:36.860251 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c989af5e-4cea-4ce3-a7dc-bdf45cdc63d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:36.953419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 732/1477] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:36.976606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463b6c7-1646-4116-81c1-7021a4f9e393 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:36.978129 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e463b6c7-1646-4116-81c1-7021a4f9e393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:37.114965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463b6c7-1646-4116-81c1-7021a4f9e393 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:37.167702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:37.211734 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280716d4-041e-4c5f-a066-dfdff78ddf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:37.212430 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280716d4-041e-4c5f-a066-dfdff78ddf7a admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:37.212702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280716d4-041e-4c5f-a066-dfdff78ddf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:37.212963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280716d4-041e-4c5f-a066-dfdff78ddf7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:37.213592 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280716d4-041e-4c5f-a066-dfdff78ddf7a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:37.215320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: WARNING py.warnings [None req-280716d4-041e-4c5f-a066-dfdff78ddf7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:20:37.215320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: warnings.warn(msg) Aug 08 02:20:37.215320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: Aug 08 02:20:37.217067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280716d4-041e-4c5f-a066-dfdff78ddf7a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:37.234944 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 746/1478] 10.222.0.39 () {56 vars in 1108 bytes} [Mon Aug 8 02:20:36 2022] GET /identity/v3/projects => generated 2553 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:20:39.232745 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec634b41-e7ed-4f46-af58-a699edd51722 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:39.233332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec634b41-e7ed-4f46-af58-a699edd51722 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:39.233332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec634b41-e7ed-4f46-af58-a699edd51722 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:39.234686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 733/1479] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:20:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:39.239891 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339112d6-3e53-4105-9cd5-88ab50d7b269 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:39.240346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339112d6-3e53-4105-9cd5-88ab50d7b269 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:39.240456 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339112d6-3e53-4105-9cd5-88ab50d7b269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:39.304976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-339112d6-3e53-4105-9cd5-88ab50d7b269 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:39.306173 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-339112d6-3e53-4105-9cd5-88ab50d7b269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:39.400188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 747/1480] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:39.444269 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec634b41-e7ed-4f46-af58-a699edd51722 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:39.445761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ec634b41-e7ed-4f46-af58-a699edd51722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:39.456095 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:39.589451 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec634b41-e7ed-4f46-af58-a699edd51722 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:39.667407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:39.700592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd11a0-3f7c-440f-b13e-a2ce0a75c598 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:39.701189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd11a0-3f7c-440f-b13e-a2ce0a75c598 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:39.701464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd11a0-3f7c-440f-b13e-a2ce0a75c598 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:39.701731 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd11a0-3f7c-440f-b13e-a2ce0a75c598 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:39.703375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-80dd11a0-3f7c-440f-b13e-a2ce0a75c598 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:39.790665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd11a0-3f7c-440f-b13e-a2ce0a75c598 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:39.792243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd11a0-3f7c-440f-b13e-a2ce0a75c598 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:39.809633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 734/1481] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:20:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:20:41.951138 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce5498-a1ae-4779-b554-85a321252bf8 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:41.951728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce5498-a1ae-4779-b554-85a321252bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:41.951728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce5498-a1ae-4779-b554-85a321252bf8 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:41.953020 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 748/1482] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:41.958057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fc052-859a-4685-8171-d90f3ba29fb0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:41.958481 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fc052-859a-4685-8171-d90f3ba29fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:41.958625 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fc052-859a-4685-8171-d90f3ba29fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:42.022008 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d07fc052-859a-4685-8171-d90f3ba29fb0 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:42.023230 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d07fc052-859a-4685-8171-d90f3ba29fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:42.068303 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:42.118632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 735/1483] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:42.154498 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce5498-a1ae-4779-b554-85a321252bf8 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:42.156055 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2bce5498-a1ae-4779-b554-85a321252bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:42.179520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:42.258213 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce5498-a1ae-4779-b554-85a321252bf8 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:42.330337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510f574-5c59-4192-b958-e0d3b414779b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:42.330940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510f574-5c59-4192-b958-e0d3b414779b service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:42.331222 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510f574-5c59-4192-b958-e0d3b414779b service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:42.331494 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510f574-5c59-4192-b958-e0d3b414779b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:42.333077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6510f574-5c59-4192-b958-e0d3b414779b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:42.407055 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:42.443394 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510f574-5c59-4192-b958-e0d3b414779b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:42.444819 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510f574-5c59-4192-b958-e0d3b414779b service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:42.468564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 749/1484] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:20:44.551537 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22e625-dbe2-40f3-8c4f-7db1436dfeec None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:44.552116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22e625-dbe2-40f3-8c4f-7db1436dfeec None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:44.552116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22e625-dbe2-40f3-8c4f-7db1436dfeec None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:44.553418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 736/1485] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:44.558550 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b54cd-f011-4c49-8b74-a99efab2e331 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:44.558979 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b54cd-f011-4c49-8b74-a99efab2e331 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:44.559089 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b54cd-f011-4c49-8b74-a99efab2e331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:44.624440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-5d4b54cd-f011-4c49-8b74-a99efab2e331 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:44.625618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5d4b54cd-f011-4c49-8b74-a99efab2e331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:44.721526 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 750/1486] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:44.760648 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22e625-dbe2-40f3-8c4f-7db1436dfeec None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:44.762175 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dc22e625-dbe2-40f3-8c4f-7db1436dfeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:44.862739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22e625-dbe2-40f3-8c4f-7db1436dfeec None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:44.932900 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:44.935576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5508be-3323-49b2-9622-762f4ba07fb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:44.936350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5508be-3323-49b2-9622-762f4ba07fb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:44.937053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5508be-3323-49b2-9622-762f4ba07fb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:44.937498 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5508be-3323-49b2-9622-762f4ba07fb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:44.939094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4f5508be-3323-49b2-9622-762f4ba07fb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:45.043125 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5508be-3323-49b2-9622-762f4ba07fb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:45.045072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5508be-3323-49b2-9622-762f4ba07fb4 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:45.063810 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 737/1487] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:20:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:20:47.201213 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958829bb-cfd5-4821-a200-c211822c82d7 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:47.201789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958829bb-cfd5-4821-a200-c211822c82d7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:47.201789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958829bb-cfd5-4821-a200-c211822c82d7 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:47.203252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 751/1488] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:47.208707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893f938-09bd-4387-b8ac-8bffe20c91ae None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:47.209120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893f938-09bd-4387-b8ac-8bffe20c91ae None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:47.209252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893f938-09bd-4387-b8ac-8bffe20c91ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:47.274441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f893f938-09bd-4387-b8ac-8bffe20c91ae None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:47.275643 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f893f938-09bd-4387-b8ac-8bffe20c91ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:47.327972 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:47.372541 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 738/1489] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:47.412862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958829bb-cfd5-4821-a200-c211822c82d7 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:47.414494 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-958829bb-cfd5-4821-a200-c211822c82d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:47.450208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:47.522625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958829bb-cfd5-4821-a200-c211822c82d7 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:47.595562 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef80a8-a20d-4f96-8b9a-55046343a901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:47.596847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef80a8-a20d-4f96-8b9a-55046343a901 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:47.596847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef80a8-a20d-4f96-8b9a-55046343a901 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:47.596847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef80a8-a20d-4f96-8b9a-55046343a901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:47.598445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-09ef80a8-a20d-4f96-8b9a-55046343a901 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:47.676816 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:47.705832 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef80a8-a20d-4f96-8b9a-55046343a901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:47.707289 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef80a8-a20d-4f96-8b9a-55046343a901 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:47.726946 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 752/1490] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:20:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:20:49.928661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6539b46-5a9a-4b33-8865-7dc1462c6158 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:49.929252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6539b46-5a9a-4b33-8865-7dc1462c6158 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:49.929252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6539b46-5a9a-4b33-8865-7dc1462c6158 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:49.930579 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 739/1491] 10.222.0.39 () {54 vars in 927 bytes} [Mon Aug 8 02:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:49.935712 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc377c-7549-4642-b4e5-09f0272fc50f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:49.936116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc377c-7549-4642-b4e5-09f0272fc50f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:49.936240 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc377c-7549-4642-b4e5-09f0272fc50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:50.001640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-52bc377c-7549-4642-b4e5-09f0272fc50f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:50.002799 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-52bc377c-7549-4642-b4e5-09f0272fc50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:50.099300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 753/1492] 10.222.0.39 () {58 vars in 1024 bytes} [Mon Aug 8 02:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:50.144367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6539b46-5a9a-4b33-8865-7dc1462c6158 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:50.145910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f6539b46-5a9a-4b33-8865-7dc1462c6158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:50.247540 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6539b46-5a9a-4b33-8865-7dc1462c6158 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:50.318586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92170761-35f1-49ed-bef6-41248a8ca1ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:50.319185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92170761-35f1-49ed-bef6-41248a8ca1ed service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:50.319465 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92170761-35f1-49ed-bef6-41248a8ca1ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:50.319731 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92170761-35f1-49ed-bef6-41248a8ca1ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:50.321304 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-92170761-35f1-49ed-bef6-41248a8ca1ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:50.330741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:50.429088 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92170761-35f1-49ed-bef6-41248a8ca1ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:50.430573 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92170761-35f1-49ed-bef6-41248a8ca1ed service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:50.449287 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 740/1493] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:20:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:21:00.097472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=82 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:21:00.446938 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=50 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:21:59.682167 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38d9cc-dc6d-4922-b532-a25d41ec104f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:59.683287 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38d9cc-dc6d-4922-b532-a25d41ec104f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:59.683287 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38d9cc-dc6d-4922-b532-a25d41ec104f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:59.760507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-0b38d9cc-dc6d-4922-b532-a25d41ec104f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:21:59.761715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0b38d9cc-dc6d-4922-b532-a25d41ec104f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:21:59.855495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 754/1494] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:21:59.863547 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79037239-c95d-4c8a-ab4a-7608d632a212 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:59.864080 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79037239-c95d-4c8a-ab4a-7608d632a212 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:59.864080 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79037239-c95d-4c8a-ab4a-7608d632a212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:59.938944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-79037239-c95d-4c8a-ab4a-7608d632a212 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:21:59.940132 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-79037239-c95d-4c8a-ab4a-7608d632a212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:22:00.033821 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 741/1495] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:22:00.038444 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38d9cc-dc6d-4922-b532-a25d41ec104f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:22:00.039664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0b38d9cc-dc6d-4922-b532-a25d41ec104f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:22:00.051715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38d9cc-dc6d-4922-b532-a25d41ec104f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:22:00.112868 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:22:00.146414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8edb8-8510-4c5f-abee-307e4c750f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:22:00.147012 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8edb8-8510-4c5f-abee-307e4c750f60 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:22:00.147285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8edb8-8510-4c5f-abee-307e4c750f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:22:00.147551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8edb8-8510-4c5f-abee-307e4c750f60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:22:00.148183 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8edb8-8510-4c5f-abee-307e4c750f60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:22:00.148481 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8edb8-8510-4c5f-abee-307e4c750f60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:22:00.149474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: WARNING py.warnings [None req-70c8edb8-8510-4c5f-abee-307e4c750f60 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:22:00.149474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: warnings.warn(msg) Aug 08 02:22:00.149474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: Aug 08 02:22:00.150383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8edb8-8510-4c5f-abee-307e4c750f60 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:22:00.161634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 755/1496] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:22:00.170586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79037239-c95d-4c8a-ab4a-7608d632a212 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:22:00.179784 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79037239-c95d-4c8a-ab4a-7608d632a212 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:22:00.240006 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:22:00.272279 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034180fd-6afd-4d67-857e-36e3c9f4b2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:22:00.272874 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034180fd-6afd-4d67-857e-36e3c9f4b2fd admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:22:00.273150 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034180fd-6afd-4d67-857e-36e3c9f4b2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:22:00.273416 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034180fd-6afd-4d67-857e-36e3c9f4b2fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:22:00.274039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034180fd-6afd-4d67-857e-36e3c9f4b2fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:22:00.275804 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-034180fd-6afd-4d67-857e-36e3c9f4b2fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:22:00.275804 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Aug 08 02:22:00.275804 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: Aug 08 02:22:00.277520 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034180fd-6afd-4d67-857e-36e3c9f4b2fd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:22:00.330952 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 742/1497] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:22:00 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:22:00.335094 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8edb8-8510-4c5f-abee-307e4c750f60 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:22:00.336596 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-70c8edb8-8510-4c5f-abee-307e4c750f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:22:00.371502 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8edb8-8510-4c5f-abee-307e4c750f60 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:22:00.464742 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56a766-e4a3-4cb6-87d5-899cc586e7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:22:00.465580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56a766-e4a3-4cb6-87d5-899cc586e7f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:22:00.465852 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56a766-e4a3-4cb6-87d5-899cc586e7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:22:00.466120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56a766-e4a3-4cb6-87d5-899cc586e7f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:22:00.466780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56a766-e4a3-4cb6-87d5-899cc586e7f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:22:00.467781 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: WARNING py.warnings [None req-7b56a766-e4a3-4cb6-87d5-899cc586e7f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:22:00.467781 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: warnings.warn(msg) Aug 08 02:22:00.467781 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: Aug 08 02:22:00.468979 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56a766-e4a3-4cb6-87d5-899cc586e7f7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:22:00.510084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 756/1498] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:22:00 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:22:00.514217 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034180fd-6afd-4d67-857e-36e3c9f4b2fd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:22:00.515793 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-034180fd-6afd-4d67-857e-36e3c9f4b2fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:22:00.579311 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034180fd-6afd-4d67-857e-36e3c9f4b2fd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:22:00.660074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:22:00.671479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05d6e8-b361-4dcf-871f-c3de12cc3462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:22:00.672077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05d6e8-b361-4dcf-871f-c3de12cc3462 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:22:00.672354 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05d6e8-b361-4dcf-871f-c3de12cc3462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:22:00.672623 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05d6e8-b361-4dcf-871f-c3de12cc3462 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:22:00.673263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05d6e8-b361-4dcf-871f-c3de12cc3462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:22:00.674292 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-9a05d6e8-b361-4dcf-871f-c3de12cc3462 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:22:00.674292 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Aug 08 02:22:00.674292 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: Aug 08 02:22:00.675180 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05d6e8-b361-4dcf-871f-c3de12cc3462 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:22:00.687957 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 743/1499] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:22:00.692016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56a766-e4a3-4cb6-87d5-899cc586e7f7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:22:00.693526 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7b56a766-e4a3-4cb6-87d5-899cc586e7f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:22:00.705034 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56a766-e4a3-4cb6-87d5-899cc586e7f7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:22:00.729306 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:22:00.798126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41e38a-200e-4682-a38a-c1855798f66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:22:00.798749 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41e38a-200e-4682-a38a-c1855798f66b admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:22:00.799026 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41e38a-200e-4682-a38a-c1855798f66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:22:00.799294 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41e38a-200e-4682-a38a-c1855798f66b admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:22:00.799931 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41e38a-200e-4682-a38a-c1855798f66b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:22:00.800924 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: WARNING py.warnings [None req-7f41e38a-200e-4682-a38a-c1855798f66b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:22:00.800924 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: warnings.warn(msg) Aug 08 02:22:00.800924 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: Aug 08 02:22:00.801803 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41e38a-200e-4682-a38a-c1855798f66b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:22:00.814705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 757/1500] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:22:00.818791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05d6e8-b361-4dcf-871f-c3de12cc3462 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:22:00.820302 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9a05d6e8-b361-4dcf-871f-c3de12cc3462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:22:00.852635 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05d6e8-b361-4dcf-871f-c3de12cc3462 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:22:00.943662 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f1a75-3fb5-4b46-a7ef-02bec5e23681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:22:00.944273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f1a75-3fb5-4b46-a7ef-02bec5e23681 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:22:00.944492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f1a75-3fb5-4b46-a7ef-02bec5e23681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:22:00.944759 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f1a75-3fb5-4b46-a7ef-02bec5e23681 admin admin] PATH_INFO: `/v3/projects/e3c5d0abcdd34037906510710479a90b/users/84c0c80c603749a9afcc57362cd56ecb/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:22:00.966007 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f1a75-3fb5-4b46-a7ef-02bec5e23681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3c5d0abcdd34037906510710479a90b, user_id=84c0c80c603749a9afcc57362cd56ecb, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:22:00.967054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-d99f1a75-3fb5-4b46-a7ef-02bec5e23681 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:22:00.967054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Aug 08 02:22:00.967054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: Aug 08 02:22:00.968981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f1a75-3fb5-4b46-a7ef-02bec5e23681 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:22:01.007058 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 744/1501] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:22:00 2022] PUT /identity/v3/projects/e3c5d0abcdd34037906510710479a90b/users/84c0c80c603749a9afcc57362cd56ecb/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:22:01.019572 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539273e-8e2b-43f0-837a-a02cbfbd35f4 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:22:01.019854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539273e-8e2b-43f0-837a-a02cbfbd35f4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:22:01.020125 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539273e-8e2b-43f0-837a-a02cbfbd35f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:22:01.076175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-4539273e-8e2b-43f0-837a-a02cbfbd35f4 None None] MFA Rules not processed for user `84c0c80c603749a9afcc57362cd56ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:22:01.077369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4539273e-8e2b-43f0-837a-a02cbfbd35f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:22:01.154297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 758/1502] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:22:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:22:01.167980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46733943-dae0-4d78-9991-46fed6a17451 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:22:01.168263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46733943-dae0-4d78-9991-46fed6a17451 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:22:01.168531 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46733943-dae0-4d78-9991-46fed6a17451 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:22:01.169861 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 745/1503] 10.222.0.39 () {54 vars in 964 bytes} [Mon Aug 8 02:22:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:22:01.174409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b56957-240c-4f9b-8256-ab262e5bae2d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:22:01.174701 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b56957-240c-4f9b-8256-ab262e5bae2d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:22:01.174970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b56957-240c-4f9b-8256-ab262e5bae2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:22:01.205471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:22:01.250386 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-65b56957-240c-4f9b-8256-ab262e5bae2d None None] MFA Rules not processed for user `2812e59d0d9948879a465fe3b5b421e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:22:01.251582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-65b56957-240c-4f9b-8256-ab262e5bae2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:22:01.331983 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 759/1504] 10.222.0.39 () {58 vars in 1061 bytes} [Mon Aug 8 02:22:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:22:01.340289 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46733943-dae0-4d78-9991-46fed6a17451 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:22:01.341801 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-46733943-dae0-4d78-9991-46fed6a17451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:22:01.393194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:22:01.447353 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46733943-dae0-4d78-9991-46fed6a17451 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:22:01.518112 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bdb3d8-5899-4f8c-99af-58b5369d2c85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:22:01.518738 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bdb3d8-5899-4f8c-99af-58b5369d2c85 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:22:01.519012 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bdb3d8-5899-4f8c-99af-58b5369d2c85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:22:01.519274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bdb3d8-5899-4f8c-99af-58b5369d2c85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:22:01.520837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-75bdb3d8-5899-4f8c-99af-58b5369d2c85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:22:01.616436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bdb3d8-5899-4f8c-99af-58b5369d2c85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:22:01.617841 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bdb3d8-5899-4f8c-99af-58b5369d2c85 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:22:01.621814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:22:01.638565 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 746/1505] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:22:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:22:01.682210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f441c-3591-417b-99bf-d78739df670a None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:22:01.682530 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f441c-3591-417b-99bf-d78739df670a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:22:01.683336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f441c-3591-417b-99bf-d78739df670a None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:22:01.684584 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 760/1506] 10.222.0.39 () {52 vars in 830 bytes} [Mon Aug 8 02:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:22:01.688612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bdb3d8-5899-4f8c-99af-58b5369d2c85 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:22:01.689807 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-75bdb3d8-5899-4f8c-99af-58b5369d2c85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:22:01.786262 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bdb3d8-5899-4f8c-99af-58b5369d2c85 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:22:01.876445 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff856d30-0771-4e76-a6fb-fa295064c545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:22:01.877054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff856d30-0771-4e76-a6fb-fa295064c545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:22:01.877329 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff856d30-0771-4e76-a6fb-fa295064c545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:22:01.877595 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff856d30-0771-4e76-a6fb-fa295064c545 admin admin] PATH_INFO: `/v3/users/84c0c80c603749a9afcc57362cd56ecb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:22:01.890131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff856d30-0771-4e76-a6fb-fa295064c545 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84c0c80c603749a9afcc57362cd56ecb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:22:01.891205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-ff856d30-0771-4e76-a6fb-fa295064c545 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:22:01.891205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Aug 08 02:22:01.891205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: Aug 08 02:22:01.892409 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff856d30-0771-4e76-a6fb-fa295064c545 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:22:01.902079 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:22:01.979523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ff856d30-0771-4e76-a6fb-fa295064c545 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84c0c80c603749a9afcc57362cd56ecb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:22:01.998274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ff856d30-0771-4e76-a6fb-fa295064c545 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84c0c80c603749a9afcc57362cd56ecb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:22:02.012373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ff856d30-0771-4e76-a6fb-fa295064c545 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84c0c80c603749a9afcc57362cd56ecb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:22:02.012909 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ff856d30-0771-4e76-a6fb-fa295064c545 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84c0c80c603749a9afcc57362cd56ecb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:22:02.013441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ff856d30-0771-4e76-a6fb-fa295064c545 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84c0c80c603749a9afcc57362cd56ecb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:22:02.040367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 747/1507] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:22:01 2022] DELETE /identity/v3/users/84c0c80c603749a9afcc57362cd56ecb => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:22:02.045271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9d539-fda8-4fc1-8243-92abcac3e55a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:22:02.045769 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9d539-fda8-4fc1-8243-92abcac3e55a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:22:02.045888 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9d539-fda8-4fc1-8243-92abcac3e55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:22:02.101776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-61f9d539-fda8-4fc1-8243-92abcac3e55a None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:22:02.102991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-61f9d539-fda8-4fc1-8243-92abcac3e55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:22:02.177167 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:22:02.191183 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 761/1508] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:22:02.201319 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff856d30-0771-4e76-a6fb-fa295064c545 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:22:02.202552 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ff856d30-0771-4e76-a6fb-fa295064c545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:22:02.305248 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff856d30-0771-4e76-a6fb-fa295064c545 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:22:02.359655 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:22:02.379270 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd344266-0372-420b-86e8-a17ef85f2f60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:22:02.379873 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd344266-0372-420b-86e8-a17ef85f2f60 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:22:02.380151 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd344266-0372-420b-86e8-a17ef85f2f60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:22:02.380420 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd344266-0372-420b-86e8-a17ef85f2f60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:22:02.382001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bd344266-0372-420b-86e8-a17ef85f2f60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:22:02.490086 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd344266-0372-420b-86e8-a17ef85f2f60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:22:02.491613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd344266-0372-420b-86e8-a17ef85f2f60 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:22:02.509990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 748/1509] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:22:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:22:02.828207 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9d539-fda8-4fc1-8243-92abcac3e55a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:22:02.829506 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-61f9d539-fda8-4fc1-8243-92abcac3e55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:22:02.840889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9d539-fda8-4fc1-8243-92abcac3e55a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:22:02.939263 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad71126-4f35-4224-9871-695b5763a52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:22:02.939929 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad71126-4f35-4224-9871-695b5763a52d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:22:02.940206 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad71126-4f35-4224-9871-695b5763a52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:22:02.940499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad71126-4f35-4224-9871-695b5763a52d admin admin] PATH_INFO: `/v3/projects/e3c5d0abcdd34037906510710479a90b` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:22:02.950288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad71126-4f35-4224-9871-695b5763a52d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3c5d0abcdd34037906510710479a90b)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:22:02.951308 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: WARNING py.warnings [None req-7ad71126-4f35-4224-9871-695b5763a52d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:22:02.951308 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: warnings.warn(msg) Aug 08 02:22:02.951308 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: Aug 08 02:22:02.952510 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad71126-4f35-4224-9871-695b5763a52d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:22:03.008336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7ad71126-4f35-4224-9871-695b5763a52d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3c5d0abcdd34037906510710479a90b', 'request_id': 'req-7ad71126-4f35-4224-9871-695b5763a52d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:22:03.014271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7ad71126-4f35-4224-9871-695b5763a52d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3c5d0abcdd34037906510710479a90b', 'request_id': 'req-7ad71126-4f35-4224-9871-695b5763a52d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:22:03.023335 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7ad71126-4f35-4224-9871-695b5763a52d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3c5d0abcdd34037906510710479a90b', 'request_id': 'req-7ad71126-4f35-4224-9871-695b5763a52d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:22:03.054955 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7ad71126-4f35-4224-9871-695b5763a52d admin admin] The token cache is being invalidate because project e3c5d0abcdd34037906510710479a90b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:22:03.055593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7ad71126-4f35-4224-9871-695b5763a52d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:22:03.055748 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7ad71126-4f35-4224-9871-695b5763a52d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:22:03.057028 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 762/1510] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:22:02 2022] DELETE /identity/v3/projects/e3c5d0abcdd34037906510710479a90b => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:22:12.507773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=62 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:22:13.052902 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:23:29.248546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7dd4c-42c8-488f-a44e-5b95c77d3d1a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:29.249151 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7dd4c-42c8-488f-a44e-5b95c77d3d1a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:29.249151 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7dd4c-42c8-488f-a44e-5b95c77d3d1a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:29.250515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:23:29.255154 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081d5c6-6619-4d9d-928b-c147345efe7a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:29.255583 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081d5c6-6619-4d9d-928b-c147345efe7a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:29.255732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081d5c6-6619-4d9d-928b-c147345efe7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:29.336132 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-0081d5c6-6619-4d9d-928b-c147345efe7a None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:29.337342 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0081d5c6-6619-4d9d-928b-c147345efe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:29.439466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:23:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:29.591910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7dd4c-42c8-488f-a44e-5b95c77d3d1a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:23:29.593950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b9c7dd4c-42c8-488f-a44e-5b95c77d3d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:29.713462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7dd4c-42c8-488f-a44e-5b95c77d3d1a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:23:29.718978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147c7cd-10df-4e8c-afbb-e9460eed6666 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:23:29.719579 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147c7cd-10df-4e8c-afbb-e9460eed6666 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:29.719860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147c7cd-10df-4e8c-afbb-e9460eed6666 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:29.720130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147c7cd-10df-4e8c-afbb-e9460eed6666 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:29.728295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147c7cd-10df-4e8c-afbb-e9460eed6666 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:23:29.730470 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147c7cd-10df-4e8c-afbb-e9460eed6666 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:23:29.739855 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 750/1513] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:23:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:23:30.560448 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0860c-30f1-4b9f-b200-c088bc2722f6 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:30.560958 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0860c-30f1-4b9f-b200-c088bc2722f6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:30.561064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0860c-30f1-4b9f-b200-c088bc2722f6 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:30.562356 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:23:30.567347 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49fb26-6f1e-4c62-8c1b-a09d47789cd6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:30.567775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49fb26-6f1e-4c62-8c1b-a09d47789cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:30.568008 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49fb26-6f1e-4c62-8c1b-a09d47789cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:30.587091 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3d49fb26-6f1e-4c62-8c1b-a09d47789cd6 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:30.588297 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3d49fb26-6f1e-4c62-8c1b-a09d47789cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:30.596269 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:30.747689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0860c-30f1-4b9f-b200-c088bc2722f6 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:23:30.761718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0860c-30f1-4b9f-b200-c088bc2722f6 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:23:30.767042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7baee-7064-468f-b53f-d0c30a420795 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:23:30.767623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7baee-7064-468f-b53f-d0c30a420795 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:30.767927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7baee-7064-468f-b53f-d0c30a420795 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:30.768195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7baee-7064-468f-b53f-d0c30a420795 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:30.776000 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7baee-7064-468f-b53f-d0c30a420795 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:23:30.778039 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7baee-7064-468f-b53f-d0c30a420795 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:23:30.787292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 765/1516] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:23:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:23:30.953085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881ffa0-d990-4ccd-9aee-d6a4f873c6c0 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:30.953364 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881ffa0-d990-4ccd-9aee-d6a4f873c6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:30.953632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881ffa0-d990-4ccd-9aee-d6a4f873c6c0 None None] PATH_INFO: `/v3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:30.954986 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 752/1517] 10.222.0.13 () {54 vars in 925 bytes} [Mon Aug 8 02:23:30 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:23:30.961489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc61a0-4419-41e2-bd51-c95f9c9a657d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:30.961791 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc61a0-4419-41e2-bd51-c95f9c9a657d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:30.962060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc61a0-4419-41e2-bd51-c95f9c9a657d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:31.019729 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-72fc61a0-4419-41e2-bd51-c95f9c9a657d None None] MFA Rules not processed for user `f62745b92a864480a9438de581e1d05e`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:31.020917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-72fc61a0-4419-41e2-bd51-c95f9c9a657d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:31.037403 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:23:31.091554 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 766/1518] 10.222.0.13 () {58 vars in 1015 bytes} [Mon Aug 8 02:23:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:40.539311 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0140fc-d220-4e8e-9415-f8586a41c965 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:40.539861 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0140fc-d220-4e8e-9415-f8586a41c965 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:40.539861 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0140fc-d220-4e8e-9415-f8586a41c965 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:40.541192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 753/1519] 10.222.0.39 () {54 vars in 961 bytes} [Mon Aug 8 02:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:23:40.546278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb665967-efab-4373-b6b0-d1e6ab63a389 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:40.546736 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb665967-efab-4373-b6b0-d1e6ab63a389 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:40.546852 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb665967-efab-4373-b6b0-d1e6ab63a389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:40.568546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-bb665967-efab-4373-b6b0-d1e6ab63a389 None None] MFA Rules not processed for user `f62745b92a864480a9438de581e1d05e`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:40.569773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bb665967-efab-4373-b6b0-d1e6ab63a389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:40.579117 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 767/1520] 10.222.0.39 () {58 vars in 1058 bytes} [Mon Aug 8 02:23:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:40.582354 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=56 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:23:40.586855 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0140fc-d220-4e8e-9415-f8586a41c965 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:23:40.600370 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0140fc-d220-4e8e-9415-f8586a41c965 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:23:40.605241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e156dc4-076e-47f4-9512-cf804a5e099e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f62745b92a864480a9438de581e1d05e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:23:40.605823 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e156dc4-076e-47f4-9512-cf804a5e099e service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:40.606096 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e156dc4-076e-47f4-9512-cf804a5e099e service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:40.606401 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e156dc4-076e-47f4-9512-cf804a5e099e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:40.614341 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e156dc4-076e-47f4-9512-cf804a5e099e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:23:40.615907 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e156dc4-076e-47f4-9512-cf804a5e099e service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:23:40.625728 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 754/1521] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:23:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:23:42.028322 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9a3d0-e38a-465b-b860-05eb9075ec4a None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:42.028908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9a3d0-e38a-465b-b860-05eb9075ec4a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:42.028908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9a3d0-e38a-465b-b860-05eb9075ec4a None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:42.030187 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:23:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:23:42.034987 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bfe79-9292-4a51-933c-66c859329102 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:42.035410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bfe79-9292-4a51-933c-66c859329102 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:42.035505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bfe79-9292-4a51-933c-66c859329102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:42.054191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ce9bfe79-9292-4a51-933c-66c859329102 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:42.055395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ce9bfe79-9292-4a51-933c-66c859329102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:42.063841 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:42.225433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9a3d0-e38a-465b-b860-05eb9075ec4a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:23:42.237556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9a3d0-e38a-465b-b860-05eb9075ec4a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:23:42.243076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15587de-7caa-40de-b9ba-65a425d5296a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:23:42.243666 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15587de-7caa-40de-b9ba-65a425d5296a service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:42.243944 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15587de-7caa-40de-b9ba-65a425d5296a service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:42.244215 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15587de-7caa-40de-b9ba-65a425d5296a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:42.252822 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15587de-7caa-40de-b9ba-65a425d5296a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:23:42.254424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15587de-7caa-40de-b9ba-65a425d5296a service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:23:42.264273 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 769/1524] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:23:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:23:43.075626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dcf9e-23d9-4118-bf1f-db4c4dc2f846 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:43.076770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dcf9e-23d9-4118-bf1f-db4c4dc2f846 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:43.077040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dcf9e-23d9-4118-bf1f-db4c4dc2f846 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:43.078435 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:23:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:23:43.083272 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb3690-0ef5-450e-866d-fc703f003406 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:43.083695 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb3690-0ef5-450e-866d-fc703f003406 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:43.083804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb3690-0ef5-450e-866d-fc703f003406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:43.103205 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7beb3690-0ef5-450e-866d-fc703f003406 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:43.104384 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7beb3690-0ef5-450e-866d-fc703f003406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:43.112705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:43.261665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5dcf9e-23d9-4118-bf1f-db4c4dc2f846 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:23:43.275959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5dcf9e-23d9-4118-bf1f-db4c4dc2f846 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:23:43.282061 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445470d-0037-4591-a4ae-d41c06d08b05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:23:43.282689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445470d-0037-4591-a4ae-d41c06d08b05 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:43.282970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445470d-0037-4591-a4ae-d41c06d08b05 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:43.283242 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445470d-0037-4591-a4ae-d41c06d08b05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:43.291415 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445470d-0037-4591-a4ae-d41c06d08b05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:23:43.293016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445470d-0037-4591-a4ae-d41c06d08b05 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:23:43.302274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 757/1527] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:23:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:23:47.434482 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b727e3-4641-47cc-8e81-23cbe569fbf4 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:47.435032 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b727e3-4641-47cc-8e81-23cbe569fbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:47.435032 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b727e3-4641-47cc-8e81-23cbe569fbf4 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:47.436354 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:23:47.441698 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f9c2a-0a67-4d31-aff6-5dcd2584e4db None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:47.442153 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f9c2a-0a67-4d31-aff6-5dcd2584e4db None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:47.442288 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f9c2a-0a67-4d31-aff6-5dcd2584e4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:47.461153 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ea6f9c2a-0a67-4d31-aff6-5dcd2584e4db None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:47.462374 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ea6f9c2a-0a67-4d31-aff6-5dcd2584e4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:47.470707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:47.543384 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b727e3-4641-47cc-8e81-23cbe569fbf4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:23:47.545145 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e5b727e3-4641-47cc-8e81-23cbe569fbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:47.622916 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b727e3-4641-47cc-8e81-23cbe569fbf4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:23:47.626428 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d58e76-caf8-4d31-8926-1dfdfb0ee34a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:23:47.627019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d58e76-caf8-4d31-8926-1dfdfb0ee34a service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:47.627293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d58e76-caf8-4d31-8926-1dfdfb0ee34a service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:47.627561 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d58e76-caf8-4d31-8926-1dfdfb0ee34a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:47.636223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d58e76-caf8-4d31-8926-1dfdfb0ee34a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:23:47.637629 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d58e76-caf8-4d31-8926-1dfdfb0ee34a service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:23:47.646805 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 772/1530] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:23:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:23:49.611317 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ce815-1a2c-4de1-8b75-073b107d3799 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:49.611875 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ce815-1a2c-4de1-8b75-073b107d3799 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:49.611875 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ce815-1a2c-4de1-8b75-073b107d3799 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:49.613180 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:23:49.618329 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc4ce1-9881-4368-829b-8057092d0508 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:49.618770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc4ce1-9881-4368-829b-8057092d0508 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:49.618875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc4ce1-9881-4368-829b-8057092d0508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:49.638740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-fccc4ce1-9881-4368-829b-8057092d0508 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:49.639931 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-fccc4ce1-9881-4368-829b-8057092d0508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:49.647657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:49.683605 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ce815-1a2c-4de1-8b75-073b107d3799 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:23:49.685278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ac1ce815-1a2c-4de1-8b75-073b107d3799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:49.699392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ce815-1a2c-4de1-8b75-073b107d3799 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:23:49.702914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62922846-72c7-4348-ad69-9b487c444fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:23:49.703513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62922846-72c7-4348-ad69-9b487c444fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:49.703795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62922846-72c7-4348-ad69-9b487c444fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:49.704091 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62922846-72c7-4348-ad69-9b487c444fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:49.714012 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62922846-72c7-4348-ad69-9b487c444fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:23:49.715492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62922846-72c7-4348-ad69-9b487c444fa7 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:23:49.724922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 760/1533] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:23:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:23:53.609124 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27687938-1be6-413d-9305-c6eed98b3d1d None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:53.609717 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27687938-1be6-413d-9305-c6eed98b3d1d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:53.609717 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27687938-1be6-413d-9305-c6eed98b3d1d None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:53.611079 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:23:53.616057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da058a59-326e-4f36-a36c-1e9401d881f1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:53.616504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da058a59-326e-4f36-a36c-1e9401d881f1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:53.616612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da058a59-326e-4f36-a36c-1e9401d881f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:53.636615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-da058a59-326e-4f36-a36c-1e9401d881f1 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:53.637801 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-da058a59-326e-4f36-a36c-1e9401d881f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:53.645522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:53.954968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27687938-1be6-413d-9305-c6eed98b3d1d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:23:53.956591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-27687938-1be6-413d-9305-c6eed98b3d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:54.019845 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:23:54.033515 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27687938-1be6-413d-9305-c6eed98b3d1d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:23:54.037016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6addc-4f57-4ac4-9d93-07389e6b2f1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:23:54.037604 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6addc-4f57-4ac4-9d93-07389e6b2f1e service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:54.037877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6addc-4f57-4ac4-9d93-07389e6b2f1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:54.038141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6addc-4f57-4ac4-9d93-07389e6b2f1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:54.047859 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6addc-4f57-4ac4-9d93-07389e6b2f1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:23:54.049318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6addc-4f57-4ac4-9d93-07389e6b2f1e service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:23:54.058662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 775/1536] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:23:56.061142 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab52c50-c0e9-42a2-b437-6a425337be55 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:56.061722 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab52c50-c0e9-42a2-b437-6a425337be55 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:56.061722 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab52c50-c0e9-42a2-b437-6a425337be55 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:56.063092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:23:56.068351 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbea13-3a5d-408d-8c70-a8b59ee7448e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:56.068747 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbea13-3a5d-408d-8c70-a8b59ee7448e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:56.068894 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbea13-3a5d-408d-8c70-a8b59ee7448e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:56.088300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-43bbea13-3a5d-408d-8c70-a8b59ee7448e None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:56.090034 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-43bbea13-3a5d-408d-8c70-a8b59ee7448e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:56.097791 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:56.396621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab52c50-c0e9-42a2-b437-6a425337be55 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:23:56.398315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7ab52c50-c0e9-42a2-b437-6a425337be55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:56.411086 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab52c50-c0e9-42a2-b437-6a425337be55 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:23:56.414601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4c811-a097-45ac-870a-1d3a25dd219c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:23:56.415188 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4c811-a097-45ac-870a-1d3a25dd219c service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:56.415462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4c811-a097-45ac-870a-1d3a25dd219c service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:56.415725 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4c811-a097-45ac-870a-1d3a25dd219c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:56.424664 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4c811-a097-45ac-870a-1d3a25dd219c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:23:56.426080 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4c811-a097-45ac-870a-1d3a25dd219c service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:23:56.436057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 763/1539] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:23:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:24:06.083696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=11 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:06.431288 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=33 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:21.179838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacef23-53b9-4267-bbc8-1b9ae74b91ec None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.180520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacef23-53b9-4267-bbc8-1b9ae74b91ec None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.180520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacef23-53b9-4267-bbc8-1b9ae74b91ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.205150 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7cacef23-53b9-4267-bbc8-1b9ae74b91ec None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:21.206397 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7cacef23-53b9-4267-bbc8-1b9ae74b91ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:21.221188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 777/1540] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:21.230587 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644986a8-e26d-4314-a47a-8ebb66f7a834 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.231832 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644986a8-e26d-4314-a47a-8ebb66f7a834 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.232327 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644986a8-e26d-4314-a47a-8ebb66f7a834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.262871 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-644986a8-e26d-4314-a47a-8ebb66f7a834 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:21.264462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-644986a8-e26d-4314-a47a-8ebb66f7a834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:21.275404 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 764/1541] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:21.280428 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cacef23-53b9-4267-bbc8-1b9ae74b91ec None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:21.297505 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cacef23-53b9-4267-bbc8-1b9ae74b91ec None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:21.306071 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef2bf1-df4e-4f2d-a02a-d92c9d553dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:21.306900 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef2bf1-df4e-4f2d-a02a-d92c9d553dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.307181 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef2bf1-df4e-4f2d-a02a-d92c9d553dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.308138 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef2bf1-df4e-4f2d-a02a-d92c9d553dd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.308801 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef2bf1-df4e-4f2d-a02a-d92c9d553dd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:21.309107 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef2bf1-df4e-4f2d-a02a-d92c9d553dd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:21.310737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef2bf1-df4e-4f2d-a02a-d92c9d553dd9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:21.327683 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 778/1542] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:21.335752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644986a8-e26d-4314-a47a-8ebb66f7a834 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:21.350779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644986a8-e26d-4314-a47a-8ebb66f7a834 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:21.359022 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45340a5d-639f-438f-ad13-3a2827701a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:21.360016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45340a5d-639f-438f-ad13-3a2827701a74 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.360517 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45340a5d-639f-438f-ad13-3a2827701a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.361003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45340a5d-639f-438f-ad13-3a2827701a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.361847 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45340a5d-639f-438f-ad13-3a2827701a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:21.364137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45340a5d-639f-438f-ad13-3a2827701a74 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:21.372218 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cc0f1-7e74-4d85-9eb0-54991a1cea43 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.372516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cc0f1-7e74-4d85-9eb0-54991a1cea43 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.372920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cc0f1-7e74-4d85-9eb0-54991a1cea43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.421563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8e3cc0f1-7e74-4d85-9eb0-54991a1cea43 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:21.422849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8e3cc0f1-7e74-4d85-9eb0-54991a1cea43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:21.436401 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 779/1543] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:25:21.448261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 765/1544] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:21.452734 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cc0f1-7e74-4d85-9eb0-54991a1cea43 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:21.458667 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd020f-0beb-4707-8e24-62fc840520e1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.460451 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd020f-0beb-4707-8e24-62fc840520e1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.460451 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd020f-0beb-4707-8e24-62fc840520e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.468825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cc0f1-7e74-4d85-9eb0-54991a1cea43 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:21.492566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d6fd020f-0beb-4707-8e24-62fc840520e1 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:21.495782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d6fd020f-0beb-4707-8e24-62fc840520e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:21.563410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a95301-530c-4015-973e-c05f430b3281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:21.564270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a95301-530c-4015-973e-c05f430b3281 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.564552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a95301-530c-4015-973e-c05f430b3281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.564825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a95301-530c-4015-973e-c05f430b3281 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.565472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a95301-530c-4015-973e-c05f430b3281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:21.567424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a95301-530c-4015-973e-c05f430b3281 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:21.581639 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 766/1545] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:25:21.585384 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa9e7d-4dfe-43f1-9015-390011226be4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.585384 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa9e7d-4dfe-43f1-9015-390011226be4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.585384 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa9e7d-4dfe-43f1-9015-390011226be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.589912 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 780/1546] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:21.591278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30858f9f-e0f4-4434-85e3-84fc1d7eaa8a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.591568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30858f9f-e0f4-4434-85e3-84fc1d7eaa8a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.591836 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30858f9f-e0f4-4434-85e3-84fc1d7eaa8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.606849 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d2fa9e7d-4dfe-43f1-9015-390011226be4 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:21.608216 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d2fa9e7d-4dfe-43f1-9015-390011226be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:21.610967 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-30858f9f-e0f4-4434-85e3-84fc1d7eaa8a None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:21.612126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-30858f9f-e0f4-4434-85e3-84fc1d7eaa8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:21.616522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 767/1547] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:21.617716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be37cc2-db15-465a-aafe-7b0825c27be1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.617994 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be37cc2-db15-465a-aafe-7b0825c27be1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.618296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be37cc2-db15-465a-aafe-7b0825c27be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.620466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 781/1548] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:21.621637 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3918a9-be87-4cfa-b10c-5139070913c5 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.621919 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3918a9-be87-4cfa-b10c-5139070913c5 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.622190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3918a9-be87-4cfa-b10c-5139070913c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.637659 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7be37cc2-db15-465a-aafe-7b0825c27be1 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:21.638883 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7be37cc2-db15-465a-aafe-7b0825c27be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:21.642232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-0c3918a9-be87-4cfa-b10c-5139070913c5 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:21.643432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0c3918a9-be87-4cfa-b10c-5139070913c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:21.646789 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 768/1549] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:21.648013 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408c758-5546-41df-96e5-0cf05cb59d9b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.648298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408c758-5546-41df-96e5-0cf05cb59d9b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.648573 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408c758-5546-41df-96e5-0cf05cb59d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.652014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 782/1550] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:21.653018 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec06b9c-0bc7-46e9-9355-28d603158bb4 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.653299 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec06b9c-0bc7-46e9-9355-28d603158bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.653570 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec06b9c-0bc7-46e9-9355-28d603158bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.672912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1408c758-5546-41df-96e5-0cf05cb59d9b None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:21.673673 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-dec06b9c-0bc7-46e9-9355-28d603158bb4 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:21.674122 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1408c758-5546-41df-96e5-0cf05cb59d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:21.674857 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dec06b9c-0bc7-46e9-9355-28d603158bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:21.682008 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 769/1551] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:21.682560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 783/1552] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:21.683312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec06b9c-0bc7-46e9-9355-28d603158bb4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:21.687181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408c758-5546-41df-96e5-0cf05cb59d9b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:21.696149 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec06b9c-0bc7-46e9-9355-28d603158bb4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:21.697902 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408c758-5546-41df-96e5-0cf05cb59d9b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:21.700746 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad738cbe-5ddb-400d-952d-a3f762c337b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:21.701334 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad738cbe-5ddb-400d-952d-a3f762c337b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.701608 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad738cbe-5ddb-400d-952d-a3f762c337b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.701874 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad738cbe-5ddb-400d-952d-a3f762c337b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.702531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad738cbe-5ddb-400d-952d-a3f762c337b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:21.702642 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179e824-1666-4d36-95c0-88f065fee650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:21.703188 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179e824-1666-4d36-95c0-88f065fee650 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.703464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179e824-1666-4d36-95c0-88f065fee650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.703735 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179e824-1666-4d36-95c0-88f065fee650 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.704103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad738cbe-5ddb-400d-952d-a3f762c337b8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:21.704356 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179e824-1666-4d36-95c0-88f065fee650 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:21.704684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179e824-1666-4d36-95c0-88f065fee650 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:21.705683 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-3179e824-1666-4d36-95c0-88f065fee650 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:25:21.705683 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Aug 08 02:25:21.705683 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: Aug 08 02:25:21.706584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179e824-1666-4d36-95c0-88f065fee650 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:21.714464 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 784/1553] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:21.715713 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b32eef-d7ac-4d21-be75-2dcc7c867500 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.715917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b32eef-d7ac-4d21-be75-2dcc7c867500 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.716171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b32eef-d7ac-4d21-be75-2dcc7c867500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.726333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 770/1554] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:21.726333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d28f5c-1249-48b2-8724-ba4f1c54c2d5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.726333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d28f5c-1249-48b2-8724-ba4f1c54c2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.726333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d28f5c-1249-48b2-8724-ba4f1c54c2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.739413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a7b32eef-d7ac-4d21-be75-2dcc7c867500 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:21.740618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a7b32eef-d7ac-4d21-be75-2dcc7c867500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:21.740736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d7d28f5c-1249-48b2-8724-ba4f1c54c2d5 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:21.742030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d7d28f5c-1249-48b2-8724-ba4f1c54c2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:21.748704 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 785/1555] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:21.749865 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b35444-98c4-4946-901b-255c61a5ba37 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.750151 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b35444-98c4-4946-901b-255c61a5ba37 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.750462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b35444-98c4-4946-901b-255c61a5ba37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.753655 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 771/1556] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:21.754847 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048dbf8-6f31-42f3-aed3-045f8d4caa7b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.755186 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048dbf8-6f31-42f3-aed3-045f8d4caa7b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.755402 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048dbf8-6f31-42f3-aed3-045f8d4caa7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.770680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-11b35444-98c4-4946-901b-255c61a5ba37 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:21.771900 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-11b35444-98c4-4946-901b-255c61a5ba37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:21.774388 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9048dbf8-6f31-42f3-aed3-045f8d4caa7b None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:21.775542 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9048dbf8-6f31-42f3-aed3-045f8d4caa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:21.780695 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 786/1557] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:21.782660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cc52e-6501-46e1-8f19-b80a8ce5df01 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.782934 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cc52e-6501-46e1-8f19-b80a8ce5df01 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.783212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cc52e-6501-46e1-8f19-b80a8ce5df01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.784671 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 772/1558] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:21.785655 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bb872-f4dc-4175-a6f1-0616fc73f4d5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.785937 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bb872-f4dc-4175-a6f1-0616fc73f4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.786207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bb872-f4dc-4175-a6f1-0616fc73f4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.805015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-842cc52e-6501-46e1-8f19-b80a8ce5df01 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:21.806204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-842cc52e-6501-46e1-8f19-b80a8ce5df01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:21.808331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c55bb872-f4dc-4175-a6f1-0616fc73f4d5 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:21.809579 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c55bb872-f4dc-4175-a6f1-0616fc73f4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:21.814519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 787/1559] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:21.815285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cc52e-6501-46e1-8f19-b80a8ce5df01 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:21.821025 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 773/1560] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:21.821025 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55bb872-f4dc-4175-a6f1-0616fc73f4d5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:21.826895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cc52e-6501-46e1-8f19-b80a8ce5df01 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:21.828542 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55bb872-f4dc-4175-a6f1-0616fc73f4d5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:21.831642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc022a-49ee-4393-92b3-a7d5890bbbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:21.832218 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc022a-49ee-4393-92b3-a7d5890bbbd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.832552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc022a-49ee-4393-92b3-a7d5890bbbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.832753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc022a-49ee-4393-92b3-a7d5890bbbd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.833052 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff966c-8554-43be-b5bb-a6a8cfc04035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:21.833377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc022a-49ee-4393-92b3-a7d5890bbbd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:21.833617 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff966c-8554-43be-b5bb-a6a8cfc04035 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.833890 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff966c-8554-43be-b5bb-a6a8cfc04035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.834155 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff966c-8554-43be-b5bb-a6a8cfc04035 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.834796 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff966c-8554-43be-b5bb-a6a8cfc04035 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:21.834991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc022a-49ee-4393-92b3-a7d5890bbbd0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:21.836664 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff966c-8554-43be-b5bb-a6a8cfc04035 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:21.845610 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 788/1561] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:21.846383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc022a-49ee-4393-92b3-a7d5890bbbd0 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:21.856303 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc022a-49ee-4393-92b3-a7d5890bbbd0 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:21.861712 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628223cd-fc76-4866-b446-749b53c2bc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:21.862305 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628223cd-fc76-4866-b446-749b53c2bc4e admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.862606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628223cd-fc76-4866-b446-749b53c2bc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.862872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628223cd-fc76-4866-b446-749b53c2bc4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.863510 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628223cd-fc76-4866-b446-749b53c2bc4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:21.863829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628223cd-fc76-4866-b446-749b53c2bc4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:21.865417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628223cd-fc76-4866-b446-749b53c2bc4e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:21.866870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 774/1562] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:21 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:21.867683 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff966c-8554-43be-b5bb-a6a8cfc04035 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:21.887799 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 789/1563] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:21.887799 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628223cd-fc76-4866-b446-749b53c2bc4e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:21.891108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628223cd-fc76-4866-b446-749b53c2bc4e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:21.922673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:21.934929 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:21.937929 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff966c-8554-43be-b5bb-a6a8cfc04035 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:21.943193 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b06f86-a6f3-4742-a429-0153a3fc356d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:21.943794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b06f86-a6f3-4742-a429-0153a3fc356d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.944079 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b06f86-a6f3-4742-a429-0153a3fc356d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.944351 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b06f86-a6f3-4742-a429-0153a3fc356d admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.945023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b06f86-a6f3-4742-a429-0153a3fc356d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:21.945329 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b06f86-a6f3-4742-a429-0153a3fc356d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:21.946988 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b06f86-a6f3-4742-a429-0153a3fc356d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:21.948177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd61a5-6801-4975-8f95-916bc78f8025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:21.948934 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd61a5-6801-4975-8f95-916bc78f8025 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.949210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd61a5-6801-4975-8f95-916bc78f8025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.949498 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd61a5-6801-4975-8f95-916bc78f8025 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.950174 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd61a5-6801-4975-8f95-916bc78f8025 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:21.950506 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd61a5-6801-4975-8f95-916bc78f8025 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:21.952066 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd61a5-6801-4975-8f95-916bc78f8025 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:21.958411 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 775/1564] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:21.959218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b06f86-a6f3-4742-a429-0153a3fc356d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:21.964042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 790/1565] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:21.965129 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd61a5-6801-4975-8f95-916bc78f8025 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:21.971117 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b06f86-a6f3-4742-a429-0153a3fc356d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:21.975789 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b88d6-3ab5-45b6-9280-bb1f82047c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:21.976384 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b88d6-3ab5-45b6-9280-bb1f82047c7e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.976660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b88d6-3ab5-45b6-9280-bb1f82047c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.976930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b88d6-3ab5-45b6-9280-bb1f82047c7e admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.977564 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b88d6-3ab5-45b6-9280-bb1f82047c7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:21.977863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b88d6-3ab5-45b6-9280-bb1f82047c7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:21.980019 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b88d6-3ab5-45b6-9280-bb1f82047c7e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:21.981530 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd61a5-6801-4975-8f95-916bc78f8025 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:21.986080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410e709-37b8-42fc-b615-a30078179e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:21.986705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410e709-37b8-42fc-b615-a30078179e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.986980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410e709-37b8-42fc-b615-a30078179e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.987255 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410e709-37b8-42fc-b615-a30078179e18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.987891 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410e709-37b8-42fc-b615-a30078179e18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:21.988206 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410e709-37b8-42fc-b615-a30078179e18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:21.989759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410e709-37b8-42fc-b615-a30078179e18 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:21.992146 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 776/1566] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:21.993026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b88d6-3ab5-45b6-9280-bb1f82047c7e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.009854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 791/1567] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:22.009854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410e709-37b8-42fc-b615-a30078179e18 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.010153 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b88d6-3ab5-45b6-9280-bb1f82047c7e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.010153 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828cc99-7f7c-4b0e-8aaf-7d13de9a732e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.010153 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828cc99-7f7c-4b0e-8aaf-7d13de9a732e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.010153 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828cc99-7f7c-4b0e-8aaf-7d13de9a732e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.010611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828cc99-7f7c-4b0e-8aaf-7d13de9a732e admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.010978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828cc99-7f7c-4b0e-8aaf-7d13de9a732e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:22.011278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828cc99-7f7c-4b0e-8aaf-7d13de9a732e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.012881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828cc99-7f7c-4b0e-8aaf-7d13de9a732e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.015908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410e709-37b8-42fc-b615-a30078179e18 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.021020 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ef9ec-9078-4bc1-a55f-ddb710053f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.021826 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ef9ec-9078-4bc1-a55f-ddb710053f3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.022104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ef9ec-9078-4bc1-a55f-ddb710053f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.022412 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ef9ec-9078-4bc1-a55f-ddb710053f3d admin admin] PATH_INFO: `/v3/projects/f673fd6851904ccbb5a916a8bdea0ab9/users/a1bb3995aade4889b7aba475f1d6d1d3/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.024188 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 777/1568] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:22.024948 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828cc99-7f7c-4b0e-8aaf-7d13de9a732e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.035555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828cc99-7f7c-4b0e-8aaf-7d13de9a732e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.036969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ef9ec-9078-4bc1-a55f-ddb710053f3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f673fd6851904ccbb5a916a8bdea0ab9, user_id=a1bb3995aade4889b7aba475f1d6d1d3, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.037986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: WARNING py.warnings [None req-720ef9ec-9078-4bc1-a55f-ddb710053f3d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:25:22.037986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: warnings.warn(msg) Aug 08 02:25:22.037986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: Aug 08 02:25:22.039947 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ef9ec-9078-4bc1-a55f-ddb710053f3d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.040247 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871981ca-96f0-4e5c-bd18-60a1d98f65f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.040830 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871981ca-96f0-4e5c-bd18-60a1d98f65f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.041113 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871981ca-96f0-4e5c-bd18-60a1d98f65f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.041382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871981ca-96f0-4e5c-bd18-60a1d98f65f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.042588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871981ca-96f0-4e5c-bd18-60a1d98f65f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.043579 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-871981ca-96f0-4e5c-bd18-60a1d98f65f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:25:22.043579 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Aug 08 02:25:22.043579 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: Aug 08 02:25:22.044809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871981ca-96f0-4e5c-bd18-60a1d98f65f1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.053365 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 792/1569] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:22 2022] PUT /identity/v3/projects/f673fd6851904ccbb5a916a8bdea0ab9/users/a1bb3995aade4889b7aba475f1d6d1d3/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:22.054143 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ef9ec-9078-4bc1-a55f-ddb710053f3d admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.077770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 778/1570] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:22 2022] POST /identity/v3/users => generated 284 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:22.078587 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871981ca-96f0-4e5c-bd18-60a1d98f65f1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.090145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871981ca-96f0-4e5c-bd18-60a1d98f65f1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.128889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ef9ec-9078-4bc1-a55f-ddb710053f3d admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.133645 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df040b-c0aa-4b39-82ad-431f69dc9f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.134270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df040b-c0aa-4b39-82ad-431f69dc9f44 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.134555 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df040b-c0aa-4b39-82ad-431f69dc9f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.134826 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df040b-c0aa-4b39-82ad-431f69dc9f44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.136209 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df040b-c0aa-4b39-82ad-431f69dc9f44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.137300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: WARNING py.warnings [None req-77df040b-c0aa-4b39-82ad-431f69dc9f44 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:25:22.137300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: warnings.warn(msg) Aug 08 02:25:22.137300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: Aug 08 02:25:22.138712 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df040b-c0aa-4b39-82ad-431f69dc9f44 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.155244 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be3b70-7033-4333-a195-5cc7d2954682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.155858 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be3b70-7033-4333-a195-5cc7d2954682 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.156179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be3b70-7033-4333-a195-5cc7d2954682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.156503 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be3b70-7033-4333-a195-5cc7d2954682 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.157245 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be3b70-7033-4333-a195-5cc7d2954682 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.159607 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be3b70-7033-4333-a195-5cc7d2954682 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.167419 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 793/1571] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:22 2022] POST /identity/v3/projects => generated 391 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:22.168348 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df040b-c0aa-4b39-82ad-431f69dc9f44 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.187357 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 779/1572] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:22 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:22.192058 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be3b70-7033-4333-a195-5cc7d2954682 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.230844 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df040b-c0aa-4b39-82ad-431f69dc9f44 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.254875 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be3b70-7033-4333-a195-5cc7d2954682 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.259447 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca72bd-b4b5-4aef-9fda-a195efa77c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.260049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca72bd-b4b5-4aef-9fda-a195efa77c5b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.260347 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca72bd-b4b5-4aef-9fda-a195efa77c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.261160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca72bd-b4b5-4aef-9fda-a195efa77c5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.261780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca72bd-b4b5-4aef-9fda-a195efa77c5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.263732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca72bd-b4b5-4aef-9fda-a195efa77c5b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.266027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:22.294777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dc1a5-ef39-47db-a248-ebecfa6500cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.295394 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dc1a5-ef39-47db-a248-ebecfa6500cd admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.295674 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dc1a5-ef39-47db-a248-ebecfa6500cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.295956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dc1a5-ef39-47db-a248-ebecfa6500cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.296579 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77dc1a5-ef39-47db-a248-ebecfa6500cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.298426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 780/1573] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:22 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:22.298563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77dc1a5-ef39-47db-a248-ebecfa6500cd admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.299306 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca72bd-b4b5-4aef-9fda-a195efa77c5b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.309636 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:22.331542 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 794/1574] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:22 2022] POST /identity/v3/projects => generated 375 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:22.331542 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dc1a5-ef39-47db-a248-ebecfa6500cd admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.368054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca72bd-b4b5-4aef-9fda-a195efa77c5b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.390055 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dc1a5-ef39-47db-a248-ebecfa6500cd admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.394813 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680568c4-eeb0-4ac3-aae4-964950c850b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.395414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680568c4-eeb0-4ac3-aae4-964950c850b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.395693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680568c4-eeb0-4ac3-aae4-964950c850b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.395963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680568c4-eeb0-4ac3-aae4-964950c850b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.396617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680568c4-eeb0-4ac3-aae4-964950c850b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.398581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680568c4-eeb0-4ac3-aae4-964950c850b0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.425170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80125344-9760-46e3-b4ed-5b2db8fa89e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.425789 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80125344-9760-46e3-b4ed-5b2db8fa89e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.426074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80125344-9760-46e3-b4ed-5b2db8fa89e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.426382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80125344-9760-46e3-b4ed-5b2db8fa89e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.427023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80125344-9760-46e3-b4ed-5b2db8fa89e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.428971 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80125344-9760-46e3-b4ed-5b2db8fa89e9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.429571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 795/1575] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:22 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:22.430990 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bd0f6-0796-4282-9016-31018a991c14 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.431442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bd0f6-0796-4282-9016-31018a991c14 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.431746 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bd0f6-0796-4282-9016-31018a991c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.459016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 781/1576] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:22 2022] POST /identity/v3/projects => generated 403 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:22.459842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80125344-9760-46e3-b4ed-5b2db8fa89e9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.466577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-bb5bd0f6-0796-4282-9016-31018a991c14 None None] MFA Rules not processed for user `a1bb3995aade4889b7aba475f1d6d1d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:22.468082 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bb5bd0f6-0796-4282-9016-31018a991c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:22.532130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80125344-9760-46e3-b4ed-5b2db8fa89e9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.536509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 796/1577] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:22.536754 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7592e3-7f40-4e31-9ec1-8feec36b96a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.537354 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7592e3-7f40-4e31-9ec1-8feec36b96a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.537470 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bd0f6-0796-4282-9016-31018a991c14 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.537629 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7592e3-7f40-4e31-9ec1-8feec36b96a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.537903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7592e3-7f40-4e31-9ec1-8feec36b96a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.538586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7592e3-7f40-4e31-9ec1-8feec36b96a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.540182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7592e3-7f40-4e31-9ec1-8feec36b96a5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.550416 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bd0f6-0796-4282-9016-31018a991c14 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.555099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 782/1578] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:22.555099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7592e3-7f40-4e31-9ec1-8feec36b96a5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.555607 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460f8a3-c184-4b1b-ac0f-9550e2cffac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.556229 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460f8a3-c184-4b1b-ac0f-9550e2cffac2 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.556510 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460f8a3-c184-4b1b-ac0f-9550e2cffac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.556779 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460f8a3-c184-4b1b-ac0f-9550e2cffac2 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.558034 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460f8a3-c184-4b1b-ac0f-9550e2cffac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.560623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460f8a3-c184-4b1b-ac0f-9550e2cffac2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.564993 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7592e3-7f40-4e31-9ec1-8feec36b96a5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.569682 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2dd14-c724-4436-b0ab-7202b0a32d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.570194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2dd14-c724-4436-b0ab-7202b0a32d3f admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.570502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2dd14-c724-4436-b0ab-7202b0a32d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.570773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2dd14-c724-4436-b0ab-7202b0a32d3f admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.571386 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2dd14-c724-4436-b0ab-7202b0a32d3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.574168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2dd14-c724-4436-b0ab-7202b0a32d3f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.586690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 797/1579] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:22 2022] POST /identity/v3/users => generated 296 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:22.587481 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460f8a3-c184-4b1b-ac0f-9550e2cffac2 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.597883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460f8a3-c184-4b1b-ac0f-9550e2cffac2 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.599414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 783/1580] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:22 2022] POST /identity/v3/users => generated 291 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:22.600308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2dd14-c724-4436-b0ab-7202b0a32d3f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.603162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c98e7d-bdcc-4dd1-b658-575e65a3dcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.603756 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c98e7d-bdcc-4dd1-b658-575e65a3dcaf admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.604035 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c98e7d-bdcc-4dd1-b658-575e65a3dcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.604839 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c98e7d-bdcc-4dd1-b658-575e65a3dcaf admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.605459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c98e7d-bdcc-4dd1-b658-575e65a3dcaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.607403 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c98e7d-bdcc-4dd1-b658-575e65a3dcaf admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.611074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2dd14-c724-4436-b0ab-7202b0a32d3f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.615671 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574c2c1-50f0-4d4e-954c-0acef3bb92f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.616290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574c2c1-50f0-4d4e-954c-0acef3bb92f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.616618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574c2c1-50f0-4d4e-954c-0acef3bb92f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.616901 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574c2c1-50f0-4d4e-954c-0acef3bb92f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.617593 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574c2c1-50f0-4d4e-954c-0acef3bb92f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.620349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574c2c1-50f0-4d4e-954c-0acef3bb92f7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.628481 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 798/1581] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:22 2022] POST /identity/v3/users => generated 287 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:22.629285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c98e7d-bdcc-4dd1-b658-575e65a3dcaf admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.637072 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:22.643237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c98e7d-bdcc-4dd1-b658-575e65a3dcaf admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.644249 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 784/1582] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:22 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:22.645511 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574c2c1-50f0-4d4e-954c-0acef3bb92f7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.648680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf23c71-27ee-48cb-9b56-66d4105b90e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.649277 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf23c71-27ee-48cb-9b56-66d4105b90e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.649556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf23c71-27ee-48cb-9b56-66d4105b90e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.649826 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf23c71-27ee-48cb-9b56-66d4105b90e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.650479 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf23c71-27ee-48cb-9b56-66d4105b90e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.650589 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:22.652616 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf23c71-27ee-48cb-9b56-66d4105b90e1 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.656709 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574c2c1-50f0-4d4e-954c-0acef3bb92f7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.661729 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde897b-fe13-4123-b7db-fb22b741fb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.662372 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde897b-fe13-4123-b7db-fb22b741fb82 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.662652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde897b-fe13-4123-b7db-fb22b741fb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.662919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde897b-fe13-4123-b7db-fb22b741fb82 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.663542 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde897b-fe13-4123-b7db-fb22b741fb82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.665561 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde897b-fe13-4123-b7db-fb22b741fb82 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.676135 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 799/1583] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:22 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:22.677168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf23c71-27ee-48cb-9b56-66d4105b90e1 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.687647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf23c71-27ee-48cb-9b56-66d4105b90e1 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.692192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 785/1584] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:22 2022] POST /identity/v3/users => generated 302 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:22.692192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f41089-8cdd-4cfb-9bef-df01b8db9ead None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.692192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f41089-8cdd-4cfb-9bef-df01b8db9ead None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.692192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f41089-8cdd-4cfb-9bef-df01b8db9ead None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.692648 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 786/1585] 10.222.0.39 () {54 vars in 962 bytes} [Mon Aug 8 02:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:22.693359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f41089-8cdd-4cfb-9bef-df01b8db9ead None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.697056 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195973b5-e6e7-44c0-8389-45c6bb118ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.697611 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195973b5-e6e7-44c0-8389-45c6bb118ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.697924 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195973b5-e6e7-44c0-8389-45c6bb118ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.698203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195973b5-e6e7-44c0-8389-45c6bb118ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.698881 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195973b5-e6e7-44c0-8389-45c6bb118ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.700481 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195973b5-e6e7-44c0-8389-45c6bb118ce2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.705981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f41089-8cdd-4cfb-9bef-df01b8db9ead None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.713790 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 800/1586] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:22.713933 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ff3e0-27b0-4a25-9280-05f5eb6ac7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.713933 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ff3e0-27b0-4a25-9280-05f5eb6ac7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.714108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195973b5-e6e7-44c0-8389-45c6bb118ce2 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.714195 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ff3e0-27b0-4a25-9280-05f5eb6ac7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.714195 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ff3e0-27b0-4a25-9280-05f5eb6ac7e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.714195 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ff3e0-27b0-4a25-9280-05f5eb6ac7e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.714195 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ff3e0-27b0-4a25-9280-05f5eb6ac7e4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.720687 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195973b5-e6e7-44c0-8389-45c6bb118ce2 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.726953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 787/1587] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:22.726953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ff3e0-27b0-4a25-9280-05f5eb6ac7e4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.727195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051de41c-d5af-41fb-ac83-4ed3607575ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.727195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051de41c-d5af-41fb-ac83-4ed3607575ae admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.727195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051de41c-d5af-41fb-ac83-4ed3607575ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.727195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051de41c-d5af-41fb-ac83-4ed3607575ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.727517 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051de41c-d5af-41fb-ac83-4ed3607575ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.728828 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051de41c-d5af-41fb-ac83-4ed3607575ae admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.734802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ff3e0-27b0-4a25-9280-05f5eb6ac7e4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.739604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2564be-d095-4109-b39d-872c291dd761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.742925 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 801/1588] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:22.743071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2564be-d095-4109-b39d-872c291dd761 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.743071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2564be-d095-4109-b39d-872c291dd761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.743071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2564be-d095-4109-b39d-872c291dd761 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.743334 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051de41c-d5af-41fb-ac83-4ed3607575ae admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.743433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2564be-d095-4109-b39d-872c291dd761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.743433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2564be-d095-4109-b39d-872c291dd761 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.750992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051de41c-d5af-41fb-ac83-4ed3607575ae admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.754336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 788/1589] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:22.755145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2564be-d095-4109-b39d-872c291dd761 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.758452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c392b-58be-4417-9606-9a805179e8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.759053 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c392b-58be-4417-9606-9a805179e8a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.759332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c392b-58be-4417-9606-9a805179e8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.759604 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c392b-58be-4417-9606-9a805179e8a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.760238 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c392b-58be-4417-9606-9a805179e8a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.761907 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c392b-58be-4417-9606-9a805179e8a7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.764820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2564be-d095-4109-b39d-872c291dd761 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.769668 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fadbd-f41c-4fae-be38-722e1cf5b93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.770331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fadbd-f41c-4fae-be38-722e1cf5b93b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.770612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fadbd-f41c-4fae-be38-722e1cf5b93b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.770880 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fadbd-f41c-4fae-be38-722e1cf5b93b admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.771507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fadbd-f41c-4fae-be38-722e1cf5b93b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.772517 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 802/1590] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:22.773234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fadbd-f41c-4fae-be38-722e1cf5b93b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.775153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c392b-58be-4417-9606-9a805179e8a7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.784760 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 789/1591] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:22.785712 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c392b-58be-4417-9606-9a805179e8a7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.786019 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48511cb5-377a-48f8-8145-61f7330581eb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.786384 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48511cb5-377a-48f8-8145-61f7330581eb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.786595 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48511cb5-377a-48f8-8145-61f7330581eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.791495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b509e-6a9b-4004-ba75-7199010b3c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.792579 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b509e-6a9b-4004-ba75-7199010b3c4b admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.792852 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b509e-6a9b-4004-ba75-7199010b3c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.793133 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b509e-6a9b-4004-ba75-7199010b3c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.794057 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b509e-6a9b-4004-ba75-7199010b3c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.796662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b509e-6a9b-4004-ba75-7199010b3c4b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.807239 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 803/1592] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:22.808582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b509e-6a9b-4004-ba75-7199010b3c4b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.818172 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b509e-6a9b-4004-ba75-7199010b3c4b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.822869 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99a2da-d367-4616-8336-4ac8a3075a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.823576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99a2da-d367-4616-8336-4ac8a3075a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.823861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99a2da-d367-4616-8336-4ac8a3075a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.824133 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99a2da-d367-4616-8336-4ac8a3075a73 admin admin] PATH_INFO: `/v3/projects/e849806325d24f3ba019c9d9882a9977/users/b6896c86755a4379b095a1088b8750b8/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.824606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-48511cb5-377a-48f8-8145-61f7330581eb None None] MFA Rules not processed for user `c4238566a9c248ffbfa314f47f02670c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:22.825794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-48511cb5-377a-48f8-8145-61f7330581eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:22.838474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99a2da-d367-4616-8336-4ac8a3075a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e849806325d24f3ba019c9d9882a9977, user_id=b6896c86755a4379b095a1088b8750b8, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.841179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99a2da-d367-4616-8336-4ac8a3075a73 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.855618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 804/1593] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:22 2022] PUT /identity/v3/projects/e849806325d24f3ba019c9d9882a9977/users/b6896c86755a4379b095a1088b8750b8/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:22.865808 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99a2da-d367-4616-8336-4ac8a3075a73 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.903133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 790/1594] 10.222.0.39 () {58 vars in 1059 bytes} [Mon Aug 8 02:25:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:22.904265 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48511cb5-377a-48f8-8145-61f7330581eb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.914362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48511cb5-377a-48f8-8145-61f7330581eb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.924003 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99a2da-d367-4616-8336-4ac8a3075a73 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.928573 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf146c2-82ce-475a-8c72-c51d43c1af7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.929165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf146c2-82ce-475a-8c72-c51d43c1af7f admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.929438 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf146c2-82ce-475a-8c72-c51d43c1af7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.929703 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf146c2-82ce-475a-8c72-c51d43c1af7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.930359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf146c2-82ce-475a-8c72-c51d43c1af7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.931936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf146c2-82ce-475a-8c72-c51d43c1af7f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.942091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 805/1595] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:22.942809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf146c2-82ce-475a-8c72-c51d43c1af7f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.951967 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf146c2-82ce-475a-8c72-c51d43c1af7f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.956593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84edc7-d959-4329-b77f-0a6a8d0a00ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.957263 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84edc7-d959-4329-b77f-0a6a8d0a00ef admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.957537 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84edc7-d959-4329-b77f-0a6a8d0a00ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.957806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84edc7-d959-4329-b77f-0a6a8d0a00ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.958475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e84edc7-d959-4329-b77f-0a6a8d0a00ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.960025 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e84edc7-d959-4329-b77f-0a6a8d0a00ef admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.973372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 806/1596] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:22.973372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84edc7-d959-4329-b77f-0a6a8d0a00ef admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.975372 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214e208-28f9-47d3-b438-f3996634cf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.975981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214e208-28f9-47d3-b438-f3996634cf9f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.976264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214e208-28f9-47d3-b438-f3996634cf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.976529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214e208-28f9-47d3-b438-f3996634cf9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.977180 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214e208-28f9-47d3-b438-f3996634cf9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.978802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214e208-28f9-47d3-b438-f3996634cf9f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.981564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84edc7-d959-4329-b77f-0a6a8d0a00ef admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.985810 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2dc88-edfb-4623-bbec-03c121d9c3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.986394 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2dc88-edfb-4623-bbec-03c121d9c3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.986668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2dc88-edfb-4623-bbec-03c121d9c3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.986933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2dc88-edfb-4623-bbec-03c121d9c3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.987538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e2dc88-edfb-4623-bbec-03c121d9c3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.989075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e2dc88-edfb-4623-bbec-03c121d9c3e7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.989462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 791/1597] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:22.990307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214e208-28f9-47d3-b438-f3996634cf9f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.000313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214e208-28f9-47d3-b438-f3996634cf9f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.005691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 807/1598] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:23.005691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2dc88-edfb-4623-bbec-03c121d9c3e7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.005923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46de85-ca6f-4099-90ac-b95555cc6879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.005923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46de85-ca6f-4099-90ac-b95555cc6879 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.006092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46de85-ca6f-4099-90ac-b95555cc6879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.006450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46de85-ca6f-4099-90ac-b95555cc6879 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.007000 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46de85-ca6f-4099-90ac-b95555cc6879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.008618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46de85-ca6f-4099-90ac-b95555cc6879 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.013007 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2dc88-edfb-4623-bbec-03c121d9c3e7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.017518 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a640eaf-81bf-439d-bb0e-2eeb21b378d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.018081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a640eaf-81bf-439d-bb0e-2eeb21b378d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.018387 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a640eaf-81bf-439d-bb0e-2eeb21b378d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.018663 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a640eaf-81bf-439d-bb0e-2eeb21b378d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.018806 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 792/1599] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:23.019278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a640eaf-81bf-439d-bb0e-2eeb21b378d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.019837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886002f-6572-4f1c-a907-016cb026199d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.020116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886002f-6572-4f1c-a907-016cb026199d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.020387 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886002f-6572-4f1c-a907-016cb026199d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.020892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a640eaf-81bf-439d-bb0e-2eeb21b378d5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.031787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 808/1600] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:23.032672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a640eaf-81bf-439d-bb0e-2eeb21b378d5 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.053464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b886002f-6572-4f1c-a907-016cb026199d None None] MFA Rules not processed for user `b6896c86755a4379b095a1088b8750b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:23.054703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b886002f-6572-4f1c-a907-016cb026199d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:23.060567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:23.077849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:23.108906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a640eaf-81bf-439d-bb0e-2eeb21b378d5 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.117575 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea81e8-96cb-4a91-9d11-210033b6ad79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.118205 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea81e8-96cb-4a91-9d11-210033b6ad79 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.118524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea81e8-96cb-4a91-9d11-210033b6ad79 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.118848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea81e8-96cb-4a91-9d11-210033b6ad79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.129800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 793/1601] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:23.130771 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886002f-6572-4f1c-a907-016cb026199d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.141917 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886002f-6572-4f1c-a907-016cb026199d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.146540 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb691046-e4b9-4719-a74b-a798e6d575e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.147183 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb691046-e4b9-4719-a74b-a798e6d575e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.147460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb691046-e4b9-4719-a74b-a798e6d575e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.147731 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb691046-e4b9-4719-a74b-a798e6d575e0 admin admin] PATH_INFO: `/v3/projects/12bf9f0d74de4a57918b688a6d969b28/users/1fd1b40cbd884915bdfac150f9718e32/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.162427 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb691046-e4b9-4719-a74b-a798e6d575e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12bf9f0d74de4a57918b688a6d969b28, user_id=1fd1b40cbd884915bdfac150f9718e32, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.165095 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb691046-e4b9-4719-a74b-a798e6d575e0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.176685 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea81e8-96cb-4a91-9d11-210033b6ad79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.178292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea81e8-96cb-4a91-9d11-210033b6ad79 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.178438 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 794/1602] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:23 2022] PUT /identity/v3/projects/12bf9f0d74de4a57918b688a6d969b28/users/1fd1b40cbd884915bdfac150f9718e32/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:23.182571 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb691046-e4b9-4719-a74b-a798e6d575e0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.187875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 809/1603] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:23.188656 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea81e8-96cb-4a91-9d11-210033b6ad79 service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.198597 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea81e8-96cb-4a91-9d11-210033b6ad79 service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.241944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb691046-e4b9-4719-a74b-a798e6d575e0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.246655 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8599b3aa-7973-436c-8066-ee1d6ae2f274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.247304 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599b3aa-7973-436c-8066-ee1d6ae2f274 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.247584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599b3aa-7973-436c-8066-ee1d6ae2f274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.247856 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599b3aa-7973-436c-8066-ee1d6ae2f274 admin admin] PATH_INFO: `/v3/projects/bc649f835360455ba6e67121cd31586e/users/405f0adce9b24b51b874b27d516a2b1f/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.257775 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e0469-75b3-442a-b194-6c5e959279f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.258454 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e0469-75b3-442a-b194-6c5e959279f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.258734 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e0469-75b3-442a-b194-6c5e959279f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.259008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e0469-75b3-442a-b194-6c5e959279f7 admin admin] PATH_INFO: `/v3/projects/3c533d28fa3f4d11b88b630a7c7bd1b4/users/43dd72b161f8442285dea411741d66b3/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.262964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8599b3aa-7973-436c-8066-ee1d6ae2f274 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc649f835360455ba6e67121cd31586e, user_id=405f0adce9b24b51b874b27d516a2b1f, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.265633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8599b3aa-7973-436c-8066-ee1d6ae2f274 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.275551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e0469-75b3-442a-b194-6c5e959279f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c533d28fa3f4d11b88b630a7c7bd1b4, user_id=43dd72b161f8442285dea411741d66b3, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.278882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e0469-75b3-442a-b194-6c5e959279f7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.279809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 795/1604] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:23 2022] PUT /identity/v3/projects/bc649f835360455ba6e67121cd31586e/users/405f0adce9b24b51b874b27d516a2b1f/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:23.280373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8599b3aa-7973-436c-8066-ee1d6ae2f274 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.305341 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 810/1605] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:23 2022] PUT /identity/v3/projects/3c533d28fa3f4d11b88b630a7c7bd1b4/users/43dd72b161f8442285dea411741d66b3/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:23.306709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e0469-75b3-442a-b194-6c5e959279f7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.347086 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8599b3aa-7973-436c-8066-ee1d6ae2f274 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.355491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:23.374711 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e0469-75b3-442a-b194-6c5e959279f7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.380144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f3e5d-e17f-4cbf-9c56-ce5a37179df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.381019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f3e5d-e17f-4cbf-9c56-ce5a37179df0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.381307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f3e5d-e17f-4cbf-9c56-ce5a37179df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.381683 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f3e5d-e17f-4cbf-9c56-ce5a37179df0 admin admin] PATH_INFO: `/v3/projects/e84c6c5ce3cd45f48c581553d10fdc10/users/3ee31a15cc0a481a85a8d0e161217df4/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.384141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:23.399819 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f3e5d-e17f-4cbf-9c56-ce5a37179df0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e84c6c5ce3cd45f48c581553d10fdc10, user_id=3ee31a15cc0a481a85a8d0e161217df4, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.403618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f3e5d-e17f-4cbf-9c56-ce5a37179df0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.422000 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 811/1606] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:23 2022] PUT /identity/v3/projects/e84c6c5ce3cd45f48c581553d10fdc10/users/3ee31a15cc0a481a85a8d0e161217df4/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:23.424852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22facf-6860-40b4-a354-071b3a8019c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.425586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22facf-6860-40b4-a354-071b3a8019c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.425864 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22facf-6860-40b4-a354-071b3a8019c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.426136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22facf-6860-40b4-a354-071b3a8019c9 admin admin] PATH_INFO: `/v3/projects/9ae7098cc9f24d9dbc4695789c5b87b4/users/17286ff2e1764ec7b91e6274c7162437/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.426566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f3e5d-e17f-4cbf-9c56-ce5a37179df0 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.442375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22facf-6860-40b4-a354-071b3a8019c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ae7098cc9f24d9dbc4695789c5b87b4, user_id=17286ff2e1764ec7b91e6274c7162437, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.445062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22facf-6860-40b4-a354-071b3a8019c9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.458943 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 796/1607] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:23 2022] PUT /identity/v3/projects/9ae7098cc9f24d9dbc4695789c5b87b4/users/17286ff2e1764ec7b91e6274c7162437/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:23.459921 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22facf-6860-40b4-a354-071b3a8019c9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.493756 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f3e5d-e17f-4cbf-9c56-ce5a37179df0 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.522115 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22facf-6860-40b4-a354-071b3a8019c9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.527593 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f3d55-406e-4d1a-87e1-601f507bef84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.528196 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f3d55-406e-4d1a-87e1-601f507bef84 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.528477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f3d55-406e-4d1a-87e1-601f507bef84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.528749 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f3d55-406e-4d1a-87e1-601f507bef84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.529366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f3d55-406e-4d1a-87e1-601f507bef84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.531294 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f3d55-406e-4d1a-87e1-601f507bef84 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.551783 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cb7a8-e0fc-4dda-b393-17f1b22d1414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.552445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cb7a8-e0fc-4dda-b393-17f1b22d1414 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.552720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cb7a8-e0fc-4dda-b393-17f1b22d1414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.552986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cb7a8-e0fc-4dda-b393-17f1b22d1414 admin admin] PATH_INFO: `/v3/projects/206cebb11b3647699555fab4e1d8ae59/users/dd93628c34e34545986a3826d51e6291/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.558516 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 797/1608] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:23 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:23.559839 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c6704-d612-4716-92bc-214ff2175dc6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.560225 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c6704-d612-4716-92bc-214ff2175dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.560618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c6704-d612-4716-92bc-214ff2175dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.568643 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cb7a8-e0fc-4dda-b393-17f1b22d1414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=206cebb11b3647699555fab4e1d8ae59, user_id=dd93628c34e34545986a3826d51e6291, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.571348 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cb7a8-e0fc-4dda-b393-17f1b22d1414 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.581369 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1e5c6704-d612-4716-92bc-214ff2175dc6 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:23.582611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1e5c6704-d612-4716-92bc-214ff2175dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:23.584994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 812/1609] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:23 2022] PUT /identity/v3/projects/206cebb11b3647699555fab4e1d8ae59/users/dd93628c34e34545986a3826d51e6291/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:23.586228 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff6df0-7e15-479f-ae26-99b78d0c73f6 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.586823 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff6df0-7e15-479f-ae26-99b78d0c73f6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.587104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff6df0-7e15-479f-ae26-99b78d0c73f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.615794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-05ff6df0-7e15-479f-ae26-99b78d0c73f6 None None] MFA Rules not processed for user `405f0adce9b24b51b874b27d516a2b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:23.617043 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-05ff6df0-7e15-479f-ae26-99b78d0c73f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:23.643445 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 798/1610] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:23.648182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57833c96-11a1-41a8-a081-7aec533fa191 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.648477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57833c96-11a1-41a8-a081-7aec533fa191 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.648758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57833c96-11a1-41a8-a081-7aec533fa191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.667390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-57833c96-11a1-41a8-a081-7aec533fa191 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:23.668663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-57833c96-11a1-41a8-a081-7aec533fa191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:23.676727 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 799/1611] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:23.680064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fba83-fabf-4d26-b96d-d53fc6576def None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.680064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fba83-fabf-4d26-b96d-d53fc6576def None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.680064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fba83-fabf-4d26-b96d-d53fc6576def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.686732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 813/1612] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:23.688080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f5a52-8d19-4ea8-ba09-8134f6bc98c7 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.688365 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f5a52-8d19-4ea8-ba09-8134f6bc98c7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.688636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f5a52-8d19-4ea8-ba09-8134f6bc98c7 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.690665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 814/1613] 10.222.0.39 () {54 vars in 912 bytes} [Mon Aug 8 02:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:23.691692 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9e7d9-3d8b-417d-94f9-d6c824cc821a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.692082 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9e7d9-3d8b-417d-94f9-d6c824cc821a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.692358 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9e7d9-3d8b-417d-94f9-d6c824cc821a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.699885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:23.702500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-369fba83-fabf-4d26-b96d-d53fc6576def None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:23.703804 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-369fba83-fabf-4d26-b96d-d53fc6576def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:23.712745 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 800/1614] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:23.713243 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-69e9e7d9-3d8b-417d-94f9-d6c824cc821a None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:23.713559 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369fba83-fabf-4d26-b96d-d53fc6576def None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.714662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-69e9e7d9-3d8b-417d-94f9-d6c824cc821a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:23.725860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369fba83-fabf-4d26-b96d-d53fc6576def None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.731428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495808b-726e-4215-8f32-a52673a116ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.732048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495808b-726e-4215-8f32-a52673a116ad admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.732368 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 815/1615] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:23.732484 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495808b-726e-4215-8f32-a52673a116ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.732663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495808b-726e-4215-8f32-a52673a116ad admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.733269 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495808b-726e-4215-8f32-a52673a116ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.733755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f02fe8-c578-44fb-b2ac-0a4baf67219b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.734066 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f02fe8-c578-44fb-b2ac-0a4baf67219b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.734469 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f02fe8-c578-44fb-b2ac-0a4baf67219b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.735257 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495808b-726e-4215-8f32-a52673a116ad admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.743378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:23.758964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 801/1616] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:23 2022] POST /identity/v3/users => generated 291 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:23.759887 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495808b-726e-4215-8f32-a52673a116ad admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.765520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a2f02fe8-c578-44fb-b2ac-0a4baf67219b None None] MFA Rules not processed for user `dd93628c34e34545986a3826d51e6291`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:23.766767 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a2f02fe8-c578-44fb-b2ac-0a4baf67219b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:23.816823 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495808b-726e-4215-8f32-a52673a116ad admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.820499 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63731b-d16c-4c9c-bbb9-dafe7ffd9f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.821104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63731b-d16c-4c9c-bbb9-dafe7ffd9f34 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.821372 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63731b-d16c-4c9c-bbb9-dafe7ffd9f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.821661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63731b-d16c-4c9c-bbb9-dafe7ffd9f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.830579 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63731b-d16c-4c9c-bbb9-dafe7ffd9f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.832006 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63731b-d16c-4c9c-bbb9-dafe7ffd9f34 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.833023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 816/1617] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:23 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:23.833859 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f02fe8-c578-44fb-b2ac-0a4baf67219b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.842776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 802/1618] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:23.842972 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f02fe8-c578-44fb-b2ac-0a4baf67219b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.843981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d424d-d770-4a30-a5f1-d42b84946549 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.844339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d424d-d770-4a30-a5f1-d42b84946549 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.844539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d424d-d770-4a30-a5f1-d42b84946549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.846660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7d540-24ee-45b7-8654-9ca9f79f77cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.847245 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7d540-24ee-45b7-8654-9ca9f79f77cf service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.847578 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7d540-24ee-45b7-8654-9ca9f79f77cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.847854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7d540-24ee-45b7-8654-9ca9f79f77cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.858965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7d540-24ee-45b7-8654-9ca9f79f77cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.861175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7d540-24ee-45b7-8654-9ca9f79f77cf service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.864848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-475d424d-d770-4a30-a5f1-d42b84946549 None None] MFA Rules not processed for user `c4238566a9c248ffbfa314f47f02670c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:23.866061 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-475d424d-d770-4a30-a5f1-d42b84946549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:23.872473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 817/1619] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:23.873773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e17335-1a36-4899-b2ae-0cab62f818d1 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.874067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e17335-1a36-4899-b2ae-0cab62f818d1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.874408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e17335-1a36-4899-b2ae-0cab62f818d1 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.875737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 818/1620] 10.222.0.39 () {54 vars in 962 bytes} [Mon Aug 8 02:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:23.879128 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e17335-1a36-4899-b2ae-0cab62f818d1 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.895996 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e17335-1a36-4899-b2ae-0cab62f818d1 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.899803 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0c724-1574-4c11-b14a-0a0415c37e5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.900433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0c724-1574-4c11-b14a-0a0415c37e5c service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.900784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0c724-1574-4c11-b14a-0a0415c37e5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.901662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0c724-1574-4c11-b14a-0a0415c37e5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.911803 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0c724-1574-4c11-b14a-0a0415c37e5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.913345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0c724-1574-4c11-b14a-0a0415c37e5c service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.925582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 819/1621] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:23.926554 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0c724-1574-4c11-b14a-0a0415c37e5c service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.935402 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0c724-1574-4c11-b14a-0a0415c37e5c service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.940755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a7454-c232-4913-956a-bd8ec40607a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.941693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a7454-c232-4913-956a-bd8ec40607a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.941822 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a7454-c232-4913-956a-bd8ec40607a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.942377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a7454-c232-4913-956a-bd8ec40607a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.954956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a7454-c232-4913-956a-bd8ec40607a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.957141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a7454-c232-4913-956a-bd8ec40607a8 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.970184 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 803/1622] 10.222.0.39 () {58 vars in 1009 bytes} [Mon Aug 8 02:25:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:23.971183 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d424d-d770-4a30-a5f1-d42b84946549 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.977512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 820/1623] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:23.978718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a7454-c232-4913-956a-bd8ec40607a8 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.984050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d424d-d770-4a30-a5f1-d42b84946549 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.991910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e2541-dc93-4133-9272-847c9866f3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.992657 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e2541-dc93-4133-9272-847c9866f3be admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.993036 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e2541-dc93-4133-9272-847c9866f3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.993540 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e2541-dc93-4133-9272-847c9866f3be admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.994153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a7454-c232-4913-956a-bd8ec40607a8 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.994497 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e2541-dc93-4133-9272-847c9866f3be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.997230 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e2541-dc93-4133-9272-847c9866f3be admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:24.000793 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d45784-0ba4-4da6-817d-49fa418ede1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:24.001604 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d45784-0ba4-4da6-817d-49fa418ede1f admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:24.001983 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d45784-0ba4-4da6-817d-49fa418ede1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:24.002477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d45784-0ba4-4da6-817d-49fa418ede1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:24.004380 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d45784-0ba4-4da6-817d-49fa418ede1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:24.007090 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d45784-0ba4-4da6-817d-49fa418ede1f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:24.011487 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 804/1624] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:24.013004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83c768-ada0-4e0e-b27f-9512fe66afe7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:24.013372 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83c768-ada0-4e0e-b27f-9512fe66afe7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:24.013990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83c768-ada0-4e0e-b27f-9512fe66afe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:24.041272 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6b83c768-ada0-4e0e-b27f-9512fe66afe7 None None] MFA Rules not processed for user `c4238566a9c248ffbfa314f47f02670c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:24.043083 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6b83c768-ada0-4e0e-b27f-9512fe66afe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:24.049860 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 821/1625] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:23 2022] POST /identity/v3/projects => generated 405 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:24.049860 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d45784-0ba4-4da6-817d-49fa418ede1f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:24.056694 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 805/1626] 10.222.0.39 () {58 vars in 1059 bytes} [Mon Aug 8 02:25:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:24.057624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83c768-ada0-4e0e-b27f-9512fe66afe7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:24.070657 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83c768-ada0-4e0e-b27f-9512fe66afe7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:24.132319 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d45784-0ba4-4da6-817d-49fa418ede1f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:24.138552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926abc2c-7626-48e0-8b20-a8fed9226fae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:24.139419 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926abc2c-7626-48e0-8b20-a8fed9226fae service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:24.139704 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926abc2c-7626-48e0-8b20-a8fed9226fae service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:24.139977 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926abc2c-7626-48e0-8b20-a8fed9226fae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:24.144859 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e12a9-4a87-4255-b898-eaa25b27c9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:24.145822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e12a9-4a87-4255-b898-eaa25b27c9b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:24.146209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e12a9-4a87-4255-b898-eaa25b27c9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:24.146608 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e12a9-4a87-4255-b898-eaa25b27c9b5 admin admin] PATH_INFO: `/v3/projects/44987f4a33c0424587fd54e6a86189aa/users/4358023bd9d0425b86aeecb90e545386/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:24.182178 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:24.189169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e12a9-4a87-4255-b898-eaa25b27c9b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44987f4a33c0424587fd54e6a86189aa, user_id=4358023bd9d0425b86aeecb90e545386, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:24.193900 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:24.194514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e12a9-4a87-4255-b898-eaa25b27c9b5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:24.216654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 806/1627] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:24 2022] PUT /identity/v3/projects/44987f4a33c0424587fd54e6a86189aa/users/4358023bd9d0425b86aeecb90e545386/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:24.220696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e12a9-4a87-4255-b898-eaa25b27c9b5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:24.236588 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926abc2c-7626-48e0-8b20-a8fed9226fae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:24.238385 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926abc2c-7626-48e0-8b20-a8fed9226fae service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:24.252319 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 822/1628] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:24.253614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926abc2c-7626-48e0-8b20-a8fed9226fae service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:24.270503 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926abc2c-7626-48e0-8b20-a8fed9226fae service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:24.304705 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e12a9-4a87-4255-b898-eaa25b27c9b5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:24.310684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14f39a-2b66-439d-910c-d426a29c8287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:24.311431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14f39a-2b66-439d-910c-d426a29c8287 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:24.311922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14f39a-2b66-439d-910c-d426a29c8287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:24.312349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14f39a-2b66-439d-910c-d426a29c8287 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:24.313442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14f39a-2b66-439d-910c-d426a29c8287 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:24.315621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14f39a-2b66-439d-910c-d426a29c8287 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:24.343948 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 807/1629] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:24 2022] POST /identity/v3/users => generated 309 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:24.344942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14f39a-2b66-439d-910c-d426a29c8287 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:24.360267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14f39a-2b66-439d-910c-d426a29c8287 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:24.366161 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798a278-0adc-42e0-8fa7-c338d2a9d259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:24.366798 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798a278-0adc-42e0-8fa7-c338d2a9d259 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:24.367129 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798a278-0adc-42e0-8fa7-c338d2a9d259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:24.367652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798a278-0adc-42e0-8fa7-c338d2a9d259 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:24.368795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798a278-0adc-42e0-8fa7-c338d2a9d259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:24.373144 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798a278-0adc-42e0-8fa7-c338d2a9d259 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:24.421514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 808/1630] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:24.421514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798a278-0adc-42e0-8fa7-c338d2a9d259 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:24.422578 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c2ed4-222e-453e-9792-55e112328efa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:24.423964 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c2ed4-222e-453e-9792-55e112328efa service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:24.424356 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c2ed4-222e-453e-9792-55e112328efa service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:24.424754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c2ed4-222e-453e-9792-55e112328efa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:24.437483 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798a278-0adc-42e0-8fa7-c338d2a9d259 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:24.444000 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e3dfc-5173-4481-81a5-acab1487ec2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:24.444766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e3dfc-5173-4481-81a5-acab1487ec2b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:24.445835 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e3dfc-5173-4481-81a5-acab1487ec2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:24.446129 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e3dfc-5173-4481-81a5-acab1487ec2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:24.447135 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e3dfc-5173-4481-81a5-acab1487ec2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:24.449794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e3dfc-5173-4481-81a5-acab1487ec2b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:24.466907 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 809/1631] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:24.468210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e3dfc-5173-4481-81a5-acab1487ec2b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:24.483173 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e3dfc-5173-4481-81a5-acab1487ec2b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:24.503690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32112865-31ea-4058-991e-e96e4e266757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:24.505363 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32112865-31ea-4058-991e-e96e4e266757 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:24.505685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32112865-31ea-4058-991e-e96e4e266757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:24.506023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32112865-31ea-4058-991e-e96e4e266757 admin admin] PATH_INFO: `/v3/projects/44987f4a33c0424587fd54e6a86189aa/users/4358023bd9d0425b86aeecb90e545386/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:24.509460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32112865-31ea-4058-991e-e96e4e266757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44987f4a33c0424587fd54e6a86189aa, user_id=4358023bd9d0425b86aeecb90e545386, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:24.515210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32112865-31ea-4058-991e-e96e4e266757 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:24.535529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 810/1632] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:24 2022] PUT /identity/v3/projects/44987f4a33c0424587fd54e6a86189aa/users/4358023bd9d0425b86aeecb90e545386/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:24.536597 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32112865-31ea-4058-991e-e96e4e266757 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:24.561982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c2ed4-222e-453e-9792-55e112328efa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:24.564042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c2ed4-222e-453e-9792-55e112328efa service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:24.601144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 823/1633] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:24.602864 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e900c49-e6bd-44e0-bd86-107b3dc8b84d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:24.603233 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e900c49-e6bd-44e0-bd86-107b3dc8b84d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:24.603585 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e900c49-e6bd-44e0-bd86-107b3dc8b84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:24.664927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-1e900c49-e6bd-44e0-bd86-107b3dc8b84d None None] MFA Rules not processed for user `4358023bd9d0425b86aeecb90e545386`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:24.666361 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1e900c49-e6bd-44e0-bd86-107b3dc8b84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:24.667311 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32112865-31ea-4058-991e-e96e4e266757 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:24.684997 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6095ddb-6f57-4209-bfac-5cbbab2cb9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:24.687280 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6095ddb-6f57-4209-bfac-5cbbab2cb9db admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:24.687672 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6095ddb-6f57-4209-bfac-5cbbab2cb9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:24.687955 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6095ddb-6f57-4209-bfac-5cbbab2cb9db admin admin] PATH_INFO: `/v3/projects/6e7579f2f45842aba0fa5fe75b32ce80/users/c7421b15e22f4718bdd12c0f4dbd688a/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:24.733208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6095ddb-6f57-4209-bfac-5cbbab2cb9db admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e7579f2f45842aba0fa5fe75b32ce80, user_id=c7421b15e22f4718bdd12c0f4dbd688a, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:24.744236 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6095ddb-6f57-4209-bfac-5cbbab2cb9db admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:24.784382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 811/1634] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:24 2022] PUT /identity/v3/projects/6e7579f2f45842aba0fa5fe75b32ce80/users/c7421b15e22f4718bdd12c0f4dbd688a/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:24.788544 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6095ddb-6f57-4209-bfac-5cbbab2cb9db admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:24.799434 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:24.847194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 824/1635] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:24.860514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e900c49-e6bd-44e0-bd86-107b3dc8b84d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:24.894413 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:24.967316 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6095ddb-6f57-4209-bfac-5cbbab2cb9db admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:24.973819 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123c743-cb0c-4e34-b224-72aecda407ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:24.974645 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123c743-cb0c-4e34-b224-72aecda407ec admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:24.974934 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123c743-cb0c-4e34-b224-72aecda407ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:24.975294 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123c743-cb0c-4e34-b224-72aecda407ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:24.976049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123c743-cb0c-4e34-b224-72aecda407ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:24.978401 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123c743-cb0c-4e34-b224-72aecda407ec admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:25.000415 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 812/1636] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:25.002277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123c743-cb0c-4e34-b224-72aecda407ec admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:25.010153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e900c49-e6bd-44e0-bd86-107b3dc8b84d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:25.024015 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123c743-cb0c-4e34-b224-72aecda407ec admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:25.036748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e3bd0-717b-472e-b113-0f62b8499b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:25.040757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e3bd0-717b-472e-b113-0f62b8499b83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:25.041053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e3bd0-717b-472e-b113-0f62b8499b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:25.041332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e3bd0-717b-472e-b113-0f62b8499b83 admin admin] PATH_INFO: `/v3/projects/6e7579f2f45842aba0fa5fe75b32ce80/users/c7421b15e22f4718bdd12c0f4dbd688a/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:25.048283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e3bd0-717b-472e-b113-0f62b8499b83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e7579f2f45842aba0fa5fe75b32ce80, user_id=c7421b15e22f4718bdd12c0f4dbd688a, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:25.059671 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e3bd0-717b-472e-b113-0f62b8499b83 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:25.092821 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 813/1637] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:25 2022] PUT /identity/v3/projects/6e7579f2f45842aba0fa5fe75b32ce80/users/c7421b15e22f4718bdd12c0f4dbd688a/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:25.104751 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40281739-70b9-46f6-83b3-7e228c1f5b9b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:25.105162 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40281739-70b9-46f6-83b3-7e228c1f5b9b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:25.105444 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40281739-70b9-46f6-83b3-7e228c1f5b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:25.194064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-40281739-70b9-46f6-83b3-7e228c1f5b9b None None] MFA Rules not processed for user `c7421b15e22f4718bdd12c0f4dbd688a`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:25.196458 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-40281739-70b9-46f6-83b3-7e228c1f5b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:25.204976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6569a-d975-4759-ae1b-ffd04bf8e261 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:25.205783 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6569a-d975-4759-ae1b-ffd04bf8e261 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:25.207094 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6569a-d975-4759-ae1b-ffd04bf8e261 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:25.210169 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6569a-d975-4759-ae1b-ffd04bf8e261 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:25.374977 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 814/1638] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:25 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:25.383773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea496dcf-e65c-4731-9ee3-ecadb519613d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:25.384334 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea496dcf-e65c-4731-9ee3-ecadb519613d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:25.386601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea496dcf-e65c-4731-9ee3-ecadb519613d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:25.391257 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 815/1639] 10.222.0.39 () {54 vars in 912 bytes} [Mon Aug 8 02:25:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:25.395847 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e3a8a-a92a-4604-afb9-360f3d0e3401 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:25.396321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e3a8a-a92a-4604-afb9-360f3d0e3401 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:25.397277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e3a8a-a92a-4604-afb9-360f3d0e3401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:25.424963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c6569a-d975-4759-ae1b-ffd04bf8e261 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:25.426636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c6569a-d975-4759-ae1b-ffd04bf8e261 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:25.429481 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c92e3a8a-a92a-4604-afb9-360f3d0e3401 None None] MFA Rules not processed for user `c4238566a9c248ffbfa314f47f02670c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:25.430906 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c92e3a8a-a92a-4604-afb9-360f3d0e3401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:25.443038 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 825/1640] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:25.444464 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6569a-d975-4759-ae1b-ffd04bf8e261 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:25.455828 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6569a-d975-4759-ae1b-ffd04bf8e261 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:25.568993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b39892-a466-499d-b603-b88b4f567a6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:25.571383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b39892-a466-499d-b603-b88b4f567a6e service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:25.573983 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b39892-a466-499d-b603-b88b4f567a6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:25.575598 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b39892-a466-499d-b603-b88b4f567a6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:25.581895 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 816/1641] 10.222.0.39 () {58 vars in 1009 bytes} [Mon Aug 8 02:25:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:25.615617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:25.628274 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b39892-a466-499d-b603-b88b4f567a6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:25.630116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b39892-a466-499d-b603-b88b4f567a6e service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:25.661337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 826/1642] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:25 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:25.831170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e3a8a-a92a-4604-afb9-360f3d0e3401 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:25.847681 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e3a8a-a92a-4604-afb9-360f3d0e3401 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:25.852159 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30512243-08f3-48f3-b7b8-ee85fcf6ae38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:25.853057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30512243-08f3-48f3-b7b8-ee85fcf6ae38 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:25.854923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30512243-08f3-48f3-b7b8-ee85fcf6ae38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:25.855482 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30512243-08f3-48f3-b7b8-ee85fcf6ae38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:25.879887 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30512243-08f3-48f3-b7b8-ee85fcf6ae38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:25.882586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30512243-08f3-48f3-b7b8-ee85fcf6ae38 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:25.899669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 817/1643] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:26.279454 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b39892-a466-499d-b603-b88b4f567a6e service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:26.467080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b39892-a466-499d-b603-b88b4f567a6e service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:26.483133 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7095d0-6d9a-4e0a-be35-eecdc7d1dd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:26.483872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7095d0-6d9a-4e0a-be35-eecdc7d1dd25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:26.484234 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7095d0-6d9a-4e0a-be35-eecdc7d1dd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:26.484514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7095d0-6d9a-4e0a-be35-eecdc7d1dd25 admin admin] PATH_INFO: `/v3/users/dd93628c34e34545986a3826d51e6291` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:26.486237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7095d0-6d9a-4e0a-be35-eecdc7d1dd25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd93628c34e34545986a3826d51e6291)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:26.487328 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: WARNING py.warnings [None req-3e7095d0-6d9a-4e0a-be35-eecdc7d1dd25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:25:26.487328 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: warnings.warn(msg) Aug 08 02:25:26.487328 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: Aug 08 02:25:26.488678 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7095d0-6d9a-4e0a-be35-eecdc7d1dd25 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:26.687023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3e7095d0-6d9a-4e0a-be35-eecdc7d1dd25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd93628c34e34545986a3826d51e6291'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:26.735544 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3e7095d0-6d9a-4e0a-be35-eecdc7d1dd25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd93628c34e34545986a3826d51e6291'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:26.762730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3e7095d0-6d9a-4e0a-be35-eecdc7d1dd25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd93628c34e34545986a3826d51e6291'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:26.762730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3e7095d0-6d9a-4e0a-be35-eecdc7d1dd25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd93628c34e34545986a3826d51e6291'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:26.762730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3e7095d0-6d9a-4e0a-be35-eecdc7d1dd25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd93628c34e34545986a3826d51e6291'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:26.820826 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 827/1644] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:26 2022] DELETE /identity/v3/users/dd93628c34e34545986a3826d51e6291 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:26.826018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b67876-a556-42a4-99ea-09daa15e0ed2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:26.826573 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b67876-a556-42a4-99ea-09daa15e0ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:26.826573 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b67876-a556-42a4-99ea-09daa15e0ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:26.895361 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-47b67876-a556-42a4-99ea-09daa15e0ed2 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:26.902514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-47b67876-a556-42a4-99ea-09daa15e0ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:26.974823 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:27.095906 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 818/1645] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:27.111063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7095d0-6d9a-4e0a-be35-eecdc7d1dd25 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:27.112510 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3e7095d0-6d9a-4e0a-be35-eecdc7d1dd25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:27.243104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7095d0-6d9a-4e0a-be35-eecdc7d1dd25 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:27.254604 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46f1b7-3f5b-42d1-ade0-6d7d7c33a099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:27.254604 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46f1b7-3f5b-42d1-ade0-6d7d7c33a099 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:27.254604 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46f1b7-3f5b-42d1-ade0-6d7d7c33a099 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:27.254604 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46f1b7-3f5b-42d1-ade0-6d7d7c33a099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:27.271940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46f1b7-3f5b-42d1-ade0-6d7d7c33a099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:27.273854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46f1b7-3f5b-42d1-ade0-6d7d7c33a099 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:27.288688 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 828/1646] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:29.155888 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b67876-a556-42a4-99ea-09daa15e0ed2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:29.159096 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-47b67876-a556-42a4-99ea-09daa15e0ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:29.196173 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b67876-a556-42a4-99ea-09daa15e0ed2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:29.208555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e406a82-b82f-44ff-9cde-c5495f5e30d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:29.209235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e406a82-b82f-44ff-9cde-c5495f5e30d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:29.209520 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e406a82-b82f-44ff-9cde-c5495f5e30d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:29.209799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e406a82-b82f-44ff-9cde-c5495f5e30d7 admin admin] PATH_INFO: `/v3/users/b6896c86755a4379b095a1088b8750b8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:29.211117 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e406a82-b82f-44ff-9cde-c5495f5e30d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6896c86755a4379b095a1088b8750b8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:29.215106 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e406a82-b82f-44ff-9cde-c5495f5e30d7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:29.307600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4e406a82-b82f-44ff-9cde-c5495f5e30d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6896c86755a4379b095a1088b8750b8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:29.320165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4e406a82-b82f-44ff-9cde-c5495f5e30d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6896c86755a4379b095a1088b8750b8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:29.345995 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4e406a82-b82f-44ff-9cde-c5495f5e30d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6896c86755a4379b095a1088b8750b8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:29.350389 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4e406a82-b82f-44ff-9cde-c5495f5e30d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6896c86755a4379b095a1088b8750b8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:29.350389 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4e406a82-b82f-44ff-9cde-c5495f5e30d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6896c86755a4379b095a1088b8750b8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:29.398283 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46f1b7-3f5b-42d1-ade0-6d7d7c33a099 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:29.398794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 819/1647] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:29 2022] DELETE /identity/v3/users/b6896c86755a4379b095a1088b8750b8 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:29.400501 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e876f8-5cef-4ada-84f5-6f0f5ef22780 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:29.401210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cc46f1b7-3f5b-42d1-ade0-6d7d7c33a099 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:29.414744 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e876f8-5cef-4ada-84f5-6f0f5ef22780 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:29.415128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e876f8-5cef-4ada-84f5-6f0f5ef22780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:29.426282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46f1b7-3f5b-42d1-ade0-6d7d7c33a099 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:29.466397 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:29.471789 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-51e876f8-5cef-4ada-84f5-6f0f5ef22780 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:29.473970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-51e876f8-5cef-4ada-84f5-6f0f5ef22780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:29.580089 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf7176-1a68-4ec8-931a-a65b4d4cfee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:29.582115 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf7176-1a68-4ec8-931a-a65b4d4cfee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:29.585278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf7176-1a68-4ec8-931a-a65b4d4cfee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:29.585789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf7176-1a68-4ec8-931a-a65b4d4cfee8 admin admin] PATH_INFO: `/v3/projects/206cebb11b3647699555fab4e1d8ae59` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:29.593428 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf7176-1a68-4ec8-931a-a65b4d4cfee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=206cebb11b3647699555fab4e1d8ae59)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:29.596481 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf7176-1a68-4ec8-931a-a65b4d4cfee8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:29.652170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 820/1648] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:29.662207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e876f8-5cef-4ada-84f5-6f0f5ef22780 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:29.668103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-51e876f8-5cef-4ada-84f5-6f0f5ef22780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:29.720923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-30cf7176-1a68-4ec8-931a-a65b4d4cfee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '206cebb11b3647699555fab4e1d8ae59', 'request_id': 'req-30cf7176-1a68-4ec8-931a-a65b4d4cfee8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:29.737649 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-30cf7176-1a68-4ec8-931a-a65b4d4cfee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '206cebb11b3647699555fab4e1d8ae59', 'request_id': 'req-30cf7176-1a68-4ec8-931a-a65b4d4cfee8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:29.762047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-30cf7176-1a68-4ec8-931a-a65b4d4cfee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '206cebb11b3647699555fab4e1d8ae59', 'request_id': 'req-30cf7176-1a68-4ec8-931a-a65b4d4cfee8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:29.769498 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:29.781044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-30cf7176-1a68-4ec8-931a-a65b4d4cfee8 admin admin] The token cache is being invalidate because project 206cebb11b3647699555fab4e1d8ae59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:29.781342 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-30cf7176-1a68-4ec8-931a-a65b4d4cfee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:29.790747 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-30cf7176-1a68-4ec8-931a-a65b4d4cfee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:29.795970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 829/1649] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:29 2022] DELETE /identity/v3/projects/206cebb11b3647699555fab4e1d8ae59 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:25:29.796873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf7176-1a68-4ec8-931a-a65b4d4cfee8 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:29.798334 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-30cf7176-1a68-4ec8-931a-a65b4d4cfee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:29.855551 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e876f8-5cef-4ada-84f5-6f0f5ef22780 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:29.953342 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf7176-1a68-4ec8-931a-a65b4d4cfee8 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:29.968436 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb41d2a-1337-472c-9ee8-a29bc15ea6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:29.969173 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb41d2a-1337-472c-9ee8-a29bc15ea6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:29.969641 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb41d2a-1337-472c-9ee8-a29bc15ea6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:29.969927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb41d2a-1337-472c-9ee8-a29bc15ea6d3 admin admin] PATH_INFO: `/v3/users/c7421b15e22f4718bdd12c0f4dbd688a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:29.971310 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb41d2a-1337-472c-9ee8-a29bc15ea6d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7421b15e22f4718bdd12c0f4dbd688a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:29.973755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb41d2a-1337-472c-9ee8-a29bc15ea6d3 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:30.003103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaceb61-4173-4eb2-9b31-321885ac0b6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:30.004022 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaceb61-4173-4eb2-9b31-321885ac0b6b service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:30.004407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaceb61-4173-4eb2-9b31-321885ac0b6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:30.004784 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaceb61-4173-4eb2-9b31-321885ac0b6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:30.009036 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-faaceb61-4173-4eb2-9b31-321885ac0b6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:30.051809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaceb61-4173-4eb2-9b31-321885ac0b6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:30.058156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaceb61-4173-4eb2-9b31-321885ac0b6b service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:30.078679 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-6fb41d2a-1337-472c-9ee8-a29bc15ea6d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7421b15e22f4718bdd12c0f4dbd688a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:30.081237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:30.084742 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 821/1650] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:25:30.097341 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-6fb41d2a-1337-472c-9ee8-a29bc15ea6d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7421b15e22f4718bdd12c0f4dbd688a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:30.129683 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-6fb41d2a-1337-472c-9ee8-a29bc15ea6d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7421b15e22f4718bdd12c0f4dbd688a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:30.131137 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-6fb41d2a-1337-472c-9ee8-a29bc15ea6d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7421b15e22f4718bdd12c0f4dbd688a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:30.131965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-6fb41d2a-1337-472c-9ee8-a29bc15ea6d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7421b15e22f4718bdd12c0f4dbd688a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:30.165366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 830/1651] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:29 2022] DELETE /identity/v3/users/c7421b15e22f4718bdd12c0f4dbd688a => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:33.519433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaceb61-4173-4eb2-9b31-321885ac0b6b service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:33.540171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaceb61-4173-4eb2-9b31-321885ac0b6b service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:33.664683 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb41d2a-1337-472c-9ee8-a29bc15ea6d3 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:33.665200 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c6f6d-d290-48f9-901d-a49fe8b2356c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:33.665678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c6f6d-d290-48f9-901d-a49fe8b2356c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:33.665947 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c6f6d-d290-48f9-901d-a49fe8b2356c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:33.666214 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6fb41d2a-1337-472c-9ee8-a29bc15ea6d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:33.666398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c6f6d-d290-48f9-901d-a49fe8b2356c admin admin] PATH_INFO: `/v3/projects/6e7579f2f45842aba0fa5fe75b32ce80` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:33.668089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c6f6d-d290-48f9-901d-a49fe8b2356c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e7579f2f45842aba0fa5fe75b32ce80)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:33.669421 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-1d5c6f6d-d290-48f9-901d-a49fe8b2356c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:25:33.669421 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Aug 08 02:25:33.669421 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: Aug 08 02:25:33.670659 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c6f6d-d290-48f9-901d-a49fe8b2356c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:33.689139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb41d2a-1337-472c-9ee8-a29bc15ea6d3 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:33.700401 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff313f-5d4f-4cd0-84bc-af4d8497a63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:33.701066 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff313f-5d4f-4cd0-84bc-af4d8497a63a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:33.701347 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff313f-5d4f-4cd0-84bc-af4d8497a63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:33.706197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff313f-5d4f-4cd0-84bc-af4d8497a63a admin admin] PATH_INFO: `/v3/projects/e849806325d24f3ba019c9d9882a9977` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:33.709260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff313f-5d4f-4cd0-84bc-af4d8497a63a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e849806325d24f3ba019c9d9882a9977)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:33.711264 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff313f-5d4f-4cd0-84bc-af4d8497a63a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:33.737340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1d5c6f6d-d290-48f9-901d-a49fe8b2356c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e7579f2f45842aba0fa5fe75b32ce80', 'request_id': 'req-1d5c6f6d-d290-48f9-901d-a49fe8b2356c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:33.746466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1d5c6f6d-d290-48f9-901d-a49fe8b2356c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e7579f2f45842aba0fa5fe75b32ce80', 'request_id': 'req-1d5c6f6d-d290-48f9-901d-a49fe8b2356c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:33.755554 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1d5c6f6d-d290-48f9-901d-a49fe8b2356c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e7579f2f45842aba0fa5fe75b32ce80', 'request_id': 'req-1d5c6f6d-d290-48f9-901d-a49fe8b2356c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:33.782036 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1d5c6f6d-d290-48f9-901d-a49fe8b2356c admin admin] The token cache is being invalidate because project 6e7579f2f45842aba0fa5fe75b32ce80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:33.782539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1d5c6f6d-d290-48f9-901d-a49fe8b2356c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:33.784017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1d5c6f6d-d290-48f9-901d-a49fe8b2356c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:33.787513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 822/1652] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:33 2022] DELETE /identity/v3/projects/6e7579f2f45842aba0fa5fe75b32ce80 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:33.793237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1f01f-b4db-4980-86fb-2dc3485fdafd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:33.793585 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1f01f-b4db-4980-86fb-2dc3485fdafd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:33.794280 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1f01f-b4db-4980-86fb-2dc3485fdafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:33.830097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b3ff313f-5d4f-4cd0-84bc-af4d8497a63a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e849806325d24f3ba019c9d9882a9977', 'request_id': 'req-b3ff313f-5d4f-4cd0-84bc-af4d8497a63a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:33.831419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-35b1f01f-b4db-4980-86fb-2dc3485fdafd None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:33.833586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-35b1f01f-b4db-4980-86fb-2dc3485fdafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:33.844158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b3ff313f-5d4f-4cd0-84bc-af4d8497a63a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e849806325d24f3ba019c9d9882a9977', 'request_id': 'req-b3ff313f-5d4f-4cd0-84bc-af4d8497a63a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:33.851505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:33.862095 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b3ff313f-5d4f-4cd0-84bc-af4d8497a63a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e849806325d24f3ba019c9d9882a9977', 'request_id': 'req-b3ff313f-5d4f-4cd0-84bc-af4d8497a63a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:33.881405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b3ff313f-5d4f-4cd0-84bc-af4d8497a63a admin admin] The token cache is being invalidate because project e849806325d24f3ba019c9d9882a9977 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:33.882177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b3ff313f-5d4f-4cd0-84bc-af4d8497a63a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:33.883378 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b3ff313f-5d4f-4cd0-84bc-af4d8497a63a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:33.885809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 831/1653] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:33 2022] DELETE /identity/v3/projects/e849806325d24f3ba019c9d9882a9977 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:25:33.889927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff313f-5d4f-4cd0-84bc-af4d8497a63a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:33.892017 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b3ff313f-5d4f-4cd0-84bc-af4d8497a63a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:33.964486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 823/1654] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:33.973689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4c43b-98c6-4736-9e26-c3a0388bedf4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:33.974511 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4c43b-98c6-4736-9e26-c3a0388bedf4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:33.974850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4c43b-98c6-4736-9e26-c3a0388bedf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:34.003288 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fce4c43b-98c6-4736-9e26-c3a0388bedf4 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:34.004506 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fce4c43b-98c6-4736-9e26-c3a0388bedf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:34.017898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff313f-5d4f-4cd0-84bc-af4d8497a63a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:34.028476 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 824/1655] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:34.043527 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4c43b-98c6-4736-9e26-c3a0388bedf4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:34.048322 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82d6e0-7516-4d66-966e-b1ed9a97b8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:34.049141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82d6e0-7516-4d66-966e-b1ed9a97b8f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:34.050932 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82d6e0-7516-4d66-966e-b1ed9a97b8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:34.051318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82d6e0-7516-4d66-966e-b1ed9a97b8f2 admin admin] PATH_INFO: `/v3/users/4358023bd9d0425b86aeecb90e545386` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:34.054763 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82d6e0-7516-4d66-966e-b1ed9a97b8f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4358023bd9d0425b86aeecb90e545386)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:34.060768 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82d6e0-7516-4d66-966e-b1ed9a97b8f2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:34.068184 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4c43b-98c6-4736-9e26-c3a0388bedf4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:34.090278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf73510-6fe7-4789-a423-8b2a34351180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:34.091373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf73510-6fe7-4789-a423-8b2a34351180 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:34.091875 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf73510-6fe7-4789-a423-8b2a34351180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:34.092330 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf73510-6fe7-4789-a423-8b2a34351180 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:34.093419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf73510-6fe7-4789-a423-8b2a34351180 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:34.094530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf73510-6fe7-4789-a423-8b2a34351180 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:34.099717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf73510-6fe7-4789-a423-8b2a34351180 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:34.129867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 825/1656] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:34.133413 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf73510-6fe7-4789-a423-8b2a34351180 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:34.177796 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf73510-6fe7-4789-a423-8b2a34351180 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:34.197295 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ef82d6e0-7516-4d66-966e-b1ed9a97b8f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4358023bd9d0425b86aeecb90e545386'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:34.200348 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a8f7d-d021-4409-8555-7d96b6d02f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:34.203268 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a8f7d-d021-4409-8555-7d96b6d02f1d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:34.206710 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a8f7d-d021-4409-8555-7d96b6d02f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:34.213667 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a8f7d-d021-4409-8555-7d96b6d02f1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:34.217097 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a8f7d-d021-4409-8555-7d96b6d02f1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:34.233206 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a8f7d-d021-4409-8555-7d96b6d02f1d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:34.237890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=81,keystone:INSERT=2,keystone:DELETE=17 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:34.249214 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ef82d6e0-7516-4d66-966e-b1ed9a97b8f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4358023bd9d0425b86aeecb90e545386'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:34.297338 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ef82d6e0-7516-4d66-966e-b1ed9a97b8f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4358023bd9d0425b86aeecb90e545386'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:34.298162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ef82d6e0-7516-4d66-966e-b1ed9a97b8f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4358023bd9d0425b86aeecb90e545386'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:34.299543 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ef82d6e0-7516-4d66-966e-b1ed9a97b8f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4358023bd9d0425b86aeecb90e545386'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:34.358349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 832/1657] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:33 2022] DELETE /identity/v3/users/4358023bd9d0425b86aeecb90e545386 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:34.362085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 826/1658] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:34 2022] POST /identity/v3/projects => generated 405 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:34.365793 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82d6e0-7516-4d66-966e-b1ed9a97b8f2 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:34.367905 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ef82d6e0-7516-4d66-966e-b1ed9a97b8f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:34.475791 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82d6e0-7516-4d66-966e-b1ed9a97b8f2 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:34.654216 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a3685-370c-4614-bb34-a367c3c66a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:34.655162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a3685-370c-4614-bb34-a367c3c66a6a admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:34.655451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a3685-370c-4614-bb34-a367c3c66a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:34.655730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a3685-370c-4614-bb34-a367c3c66a6a admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:34.658298 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a3685-370c-4614-bb34-a367c3c66a6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:34.659949 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a3685-370c-4614-bb34-a367c3c66a6a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:34.738998 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a8f7d-d021-4409-8555-7d96b6d02f1d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:34.749561 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 833/1659] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:34 2022] POST /identity/v3/users => generated 303 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:25:34.753530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a8f7d-d021-4409-8555-7d96b6d02f1d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:34.788330 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a93b0b-4aa9-4369-a009-5b59adddc342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:34.792507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a93b0b-4aa9-4369-a009-5b59adddc342 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:34.792785 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a93b0b-4aa9-4369-a009-5b59adddc342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:34.795241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a93b0b-4aa9-4369-a009-5b59adddc342 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:34.799126 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a93b0b-4aa9-4369-a009-5b59adddc342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:34.803851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a93b0b-4aa9-4369-a009-5b59adddc342 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:34.824317 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a152539-fbc9-4560-a7e7-8c6cb91570a9 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:34.824608 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a152539-fbc9-4560-a7e7-8c6cb91570a9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:34.824872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a152539-fbc9-4560-a7e7-8c6cb91570a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:34.858921 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 827/1660] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:34.860822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a93b0b-4aa9-4369-a009-5b59adddc342 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:34.890165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a93b0b-4aa9-4369-a009-5b59adddc342 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:34.899302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-4a152539-fbc9-4560-a7e7-8c6cb91570a9 None None] MFA Rules not processed for user `43dd72b161f8442285dea411741d66b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:34.900588 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4a152539-fbc9-4560-a7e7-8c6cb91570a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:34.909194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f57c4-bb22-45b1-860b-44021d78d4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:34.909525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f57c4-bb22-45b1-860b-44021d78d4ee admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:34.909767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f57c4-bb22-45b1-860b-44021d78d4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:34.910039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f57c4-bb22-45b1-860b-44021d78d4ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:34.910717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f57c4-bb22-45b1-860b-44021d78d4ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:34.917209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f57c4-bb22-45b1-860b-44021d78d4ee admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:34.942204 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 828/1661] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:34.947523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f57c4-bb22-45b1-860b-44021d78d4ee admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:34.977070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f57c4-bb22-45b1-860b-44021d78d4ee admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:34.988138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266c585-5a8f-4d49-9785-31fea3fce4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:34.990712 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266c585-5a8f-4d49-9785-31fea3fce4d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:34.992517 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266c585-5a8f-4d49-9785-31fea3fce4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:34.995349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266c585-5a8f-4d49-9785-31fea3fce4d4 admin admin] PATH_INFO: `/v3/projects/b0723e1594f94cb0a0e694b863e920fd/users/46ecdd9d0eb84701a651a1c687ad4522/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:35.033729 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266c585-5a8f-4d49-9785-31fea3fce4d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0723e1594f94cb0a0e694b863e920fd, user_id=46ecdd9d0eb84701a651a1c687ad4522, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:35.042049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266c585-5a8f-4d49-9785-31fea3fce4d4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:35.060095 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:35.087723 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 829/1662] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:34 2022] PUT /identity/v3/projects/b0723e1594f94cb0a0e694b863e920fd/users/46ecdd9d0eb84701a651a1c687ad4522/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:35.093474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 834/1663] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:35.102353 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a152539-fbc9-4560-a7e7-8c6cb91570a9 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:35.102658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccd30a-2351-4d05-9443-3a84318a8950 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:35.102944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccd30a-2351-4d05-9443-3a84318a8950 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:35.103217 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccd30a-2351-4d05-9443-3a84318a8950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:35.104547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4a152539-fbc9-4560-a7e7-8c6cb91570a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:35.138420 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4cccd30a-2351-4d05-9443-3a84318a8950 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:35.142266 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4cccd30a-2351-4d05-9443-3a84318a8950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:35.290129 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 830/1664] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:35.311785 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a152539-fbc9-4560-a7e7-8c6cb91570a9 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:35.317544 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccd30a-2351-4d05-9443-3a84318a8950 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:35.318934 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4cccd30a-2351-4d05-9443-3a84318a8950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:35.370713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccd30a-2351-4d05-9443-3a84318a8950 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:35.382624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0991c-5313-4959-bd9f-cc82519930dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:35.383764 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0991c-5313-4959-bd9f-cc82519930dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:35.385513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0991c-5313-4959-bd9f-cc82519930dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:35.388661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0991c-5313-4959-bd9f-cc82519930dd admin admin] PATH_INFO: `/v3/projects/44987f4a33c0424587fd54e6a86189aa` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:35.392283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0991c-5313-4959-bd9f-cc82519930dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=44987f4a33c0424587fd54e6a86189aa)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:35.398600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0991c-5313-4959-bd9f-cc82519930dd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:35.478096 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:35.496066 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11453c-75bd-4733-b844-3923f5d8179e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:35.496858 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11453c-75bd-4733-b844-3923f5d8179e service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:35.497991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11453c-75bd-4733-b844-3923f5d8179e service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:35.497991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11453c-75bd-4733-b844-3923f5d8179e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:35.537500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-39a0991c-5313-4959-bd9f-cc82519930dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44987f4a33c0424587fd54e6a86189aa', 'request_id': 'req-39a0991c-5313-4959-bd9f-cc82519930dd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:35.558892 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-39a0991c-5313-4959-bd9f-cc82519930dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44987f4a33c0424587fd54e6a86189aa', 'request_id': 'req-39a0991c-5313-4959-bd9f-cc82519930dd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:35.581172 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-39a0991c-5313-4959-bd9f-cc82519930dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44987f4a33c0424587fd54e6a86189aa', 'request_id': 'req-39a0991c-5313-4959-bd9f-cc82519930dd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:35.607588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-39a0991c-5313-4959-bd9f-cc82519930dd admin admin] The token cache is being invalidate because project 44987f4a33c0424587fd54e6a86189aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:35.608347 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-39a0991c-5313-4959-bd9f-cc82519930dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:35.610827 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-39a0991c-5313-4959-bd9f-cc82519930dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:35.612951 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 831/1665] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:35 2022] DELETE /identity/v3/projects/44987f4a33c0424587fd54e6a86189aa => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:25:35.613509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0991c-5313-4959-bd9f-cc82519930dd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:35.618700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-39a0991c-5313-4959-bd9f-cc82519930dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:35.648962 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11453c-75bd-4733-b844-3923f5d8179e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:35.651081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11453c-75bd-4733-b844-3923f5d8179e service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:35.663121 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 835/1666] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:25:35.665107 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11453c-75bd-4733-b844-3923f5d8179e service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:35.666750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8c11453c-75bd-4733-b844-3923f5d8179e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:35.688134 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11453c-75bd-4733-b844-3923f5d8179e service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:35.727823 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:35.751638 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0991c-5313-4959-bd9f-cc82519930dd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:35.758772 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a39986-bb50-42b9-87aa-84dcf8ab12c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:35.758772 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a39986-bb50-42b9-87aa-84dcf8ab12c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:35.758772 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a39986-bb50-42b9-87aa-84dcf8ab12c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:35.758772 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a39986-bb50-42b9-87aa-84dcf8ab12c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:35.763206 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-32a39986-bb50-42b9-87aa-84dcf8ab12c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:35.767498 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a45fe7-55ba-43a1-a557-c7a28d091f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:35.768318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a45fe7-55ba-43a1-a557-c7a28d091f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:35.768727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a45fe7-55ba-43a1-a557-c7a28d091f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:35.769074 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a45fe7-55ba-43a1-a557-c7a28d091f01 admin admin] PATH_INFO: `/v3/users/405f0adce9b24b51b874b27d516a2b1f` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:35.770836 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a45fe7-55ba-43a1-a557-c7a28d091f01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=405f0adce9b24b51b874b27d516a2b1f)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:35.776171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a45fe7-55ba-43a1-a557-c7a28d091f01 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:35.792202 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a39986-bb50-42b9-87aa-84dcf8ab12c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:35.795139 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a39986-bb50-42b9-87aa-84dcf8ab12c6 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:35.832598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 832/1667] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:35.834694 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f46df-f231-4e0e-b214-75f375350ce3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:35.835263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f46df-f231-4e0e-b214-75f375350ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:35.837626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f46df-f231-4e0e-b214-75f375350ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:35.901778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-33a45fe7-55ba-43a1-a557-c7a28d091f01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '405f0adce9b24b51b874b27d516a2b1f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:35.908844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-909f46df-f231-4e0e-b214-75f375350ce3 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:35.917484 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-909f46df-f231-4e0e-b214-75f375350ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:35.924390 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-33a45fe7-55ba-43a1-a557-c7a28d091f01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '405f0adce9b24b51b874b27d516a2b1f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:35.945539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 833/1668] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:35.947136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2b593-2ba0-4846-9fca-5a027700d84b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:35.947714 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2b593-2ba0-4846-9fca-5a027700d84b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:35.947989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2b593-2ba0-4846-9fca-5a027700d84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:35.950097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-33a45fe7-55ba-43a1-a557-c7a28d091f01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '405f0adce9b24b51b874b27d516a2b1f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:35.951217 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-33a45fe7-55ba-43a1-a557-c7a28d091f01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '405f0adce9b24b51b874b27d516a2b1f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:35.952179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-33a45fe7-55ba-43a1-a557-c7a28d091f01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '405f0adce9b24b51b874b27d516a2b1f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:35.977775 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 836/1669] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:35 2022] DELETE /identity/v3/users/405f0adce9b24b51b874b27d516a2b1f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:35.980185 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5f7bd-0ec0-473e-acb8-0d72ffaa29a5 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:35.980744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5f7bd-0ec0-473e-acb8-0d72ffaa29a5 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:35.981687 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5f7bd-0ec0-473e-acb8-0d72ffaa29a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:36.013902 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-57d2b593-2ba0-4846-9fca-5a027700d84b None None] MFA Rules not processed for user `17286ff2e1764ec7b91e6274c7162437`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:36.015171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-57d2b593-2ba0-4846-9fca-5a027700d84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:36.016322 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-64c5f7bd-0ec0-473e-acb8-0d72ffaa29a5 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:36.017735 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-64c5f7bd-0ec0-473e-acb8-0d72ffaa29a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:36.126899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 837/1670] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:36.126899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a2d84-5390-4367-b70c-5618fb686087 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:36.126899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a2d84-5390-4367-b70c-5618fb686087 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:36.126899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a2d84-5390-4367-b70c-5618fb686087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:36.168310 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 834/1671] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:36.177541 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d42a2d84-5390-4367-b70c-5618fb686087 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:36.178815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d42a2d84-5390-4367-b70c-5618fb686087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:36.181191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2b593-2ba0-4846-9fca-5a027700d84b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:36.199514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 838/1672] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:36.203264 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a2d84-5390-4367-b70c-5618fb686087 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:36.204921 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2b593-2ba0-4846-9fca-5a027700d84b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:36.207107 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d42a2d84-5390-4367-b70c-5618fb686087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:36.215688 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac1ff9-bb51-465f-86bb-e65845831ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:36.215980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac1ff9-bb51-465f-86bb-e65845831ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:36.217128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac1ff9-bb51-465f-86bb-e65845831ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:36.217415 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac1ff9-bb51-465f-86bb-e65845831ad3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:36.218079 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac1ff9-bb51-465f-86bb-e65845831ad3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:36.220197 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac1ff9-bb51-465f-86bb-e65845831ad3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:36.221883 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac1ff9-bb51-465f-86bb-e65845831ad3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:36.252628 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 835/1673] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:36.253839 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac1ff9-bb51-465f-86bb-e65845831ad3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:36.273577 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac1ff9-bb51-465f-86bb-e65845831ad3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:36.297797 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:36.299653 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c261347-15d4-484c-b3f8-5ac74f275a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:36.300418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c261347-15d4-484c-b3f8-5ac74f275a62 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:36.301184 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c261347-15d4-484c-b3f8-5ac74f275a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:36.302273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c261347-15d4-484c-b3f8-5ac74f275a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:36.305220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c261347-15d4-484c-b3f8-5ac74f275a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:36.309192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c261347-15d4-484c-b3f8-5ac74f275a62 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:36.342721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a2d84-5390-4367-b70c-5618fb686087 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:36.363852 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e55fb8-b4dd-43e2-ba1c-012cf693a78f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:36.364489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e55fb8-b4dd-43e2-ba1c-012cf693a78f service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:36.364767 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e55fb8-b4dd-43e2-ba1c-012cf693a78f service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:36.365039 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e55fb8-b4dd-43e2-ba1c-012cf693a78f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:36.377859 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 836/1674] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:36 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:36.379005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c261347-15d4-484c-b3f8-5ac74f275a62 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:36.381124 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8c261347-15d4-484c-b3f8-5ac74f275a62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:36.389142 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:36.391560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e55fb8-b4dd-43e2-ba1c-012cf693a78f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:36.405635 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e55fb8-b4dd-43e2-ba1c-012cf693a78f service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:36.430266 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 839/1675] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:36.431125 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e55fb8-b4dd-43e2-ba1c-012cf693a78f service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:36.467212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e55fb8-b4dd-43e2-ba1c-012cf693a78f service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:36.550930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c261347-15d4-484c-b3f8-5ac74f275a62 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:36.562169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2db77-6451-4318-b32f-5bac834ffb8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:36.563003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2db77-6451-4318-b32f-5bac834ffb8b service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:36.563352 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2db77-6451-4318-b32f-5bac834ffb8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:36.563700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2db77-6451-4318-b32f-5bac834ffb8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:36.646016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa26f56-54c8-411c-a5e2-463899506ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:36.646661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa26f56-54c8-411c-a5e2-463899506ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:36.646938 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa26f56-54c8-411c-a5e2-463899506ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:36.647211 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa26f56-54c8-411c-a5e2-463899506ceb admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:36.647855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa26f56-54c8-411c-a5e2-463899506ceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:36.649794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa26f56-54c8-411c-a5e2-463899506ceb admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:36.696985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae2db77-6451-4318-b32f-5bac834ffb8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:36.698703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae2db77-6451-4318-b32f-5bac834ffb8b service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:36.709720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 840/1676] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:36 2022] POST /identity/v3/users => generated 292 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:36.714156 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa26f56-54c8-411c-a5e2-463899506ceb admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:36.734772 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa26f56-54c8-411c-a5e2-463899506ceb admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:36.737251 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 837/1677] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:36.758881 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e4881-fba6-4f94-8c8f-3eb2bc32c2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:36.759516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e4881-fba6-4f94-8c8f-3eb2bc32c2d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:36.759799 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e4881-fba6-4f94-8c8f-3eb2bc32c2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:36.760943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e4881-fba6-4f94-8c8f-3eb2bc32c2d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:36.761242 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e4881-fba6-4f94-8c8f-3eb2bc32c2d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:36.764459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e4881-fba6-4f94-8c8f-3eb2bc32c2d7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:36.796238 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 841/1678] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:36.800387 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2db77-6451-4318-b32f-5bac834ffb8b service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:36.820469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2db77-6451-4318-b32f-5bac834ffb8b service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:36.842742 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6421a4-a1f9-43ba-a9eb-8e327ba0a44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:36.843383 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6421a4-a1f9-43ba-a9eb-8e327ba0a44e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:36.843666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6421a4-a1f9-43ba-a9eb-8e327ba0a44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:36.843922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6421a4-a1f9-43ba-a9eb-8e327ba0a44e admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:36.844597 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6421a4-a1f9-43ba-a9eb-8e327ba0a44e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:36.846814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6421a4-a1f9-43ba-a9eb-8e327ba0a44e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:36.871196 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e4881-fba6-4f94-8c8f-3eb2bc32c2d7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:36.892525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e4881-fba6-4f94-8c8f-3eb2bc32c2d7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:36.904082 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5efb08-7613-48c9-b73f-8cf69ab2bb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:36.904767 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5efb08-7613-48c9-b73f-8cf69ab2bb47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:36.905066 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5efb08-7613-48c9-b73f-8cf69ab2bb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:36.905330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5efb08-7613-48c9-b73f-8cf69ab2bb47 admin admin] PATH_INFO: `/v3/projects/911b7361e9064f0eafe6fa1328042556/users/e69ec152e64b4447a314a3db5c047412/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:36.909881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 838/1679] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:36.959195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5efb08-7613-48c9-b73f-8cf69ab2bb47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=911b7361e9064f0eafe6fa1328042556, user_id=e69ec152e64b4447a314a3db5c047412, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:36.961914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5efb08-7613-48c9-b73f-8cf69ab2bb47 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:37.002615 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 842/1680] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:36 2022] PUT /identity/v3/projects/911b7361e9064f0eafe6fa1328042556/users/e69ec152e64b4447a314a3db5c047412/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:37.010340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1401fa-b9eb-41d2-8a99-061c7d52b421 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:37.015967 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1401fa-b9eb-41d2-8a99-061c7d52b421 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:37.019049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1401fa-b9eb-41d2-8a99-061c7d52b421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:37.119074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6b1401fa-b9eb-41d2-8a99-061c7d52b421 None None] MFA Rules not processed for user `e69ec152e64b4447a314a3db5c047412`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:37.127623 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6b1401fa-b9eb-41d2-8a99-061c7d52b421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:37.318538 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 839/1681] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:37.319989 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5efb08-7613-48c9-b73f-8cf69ab2bb47 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:37.518036 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5efb08-7613-48c9-b73f-8cf69ab2bb47 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:37.551883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1947b5-5815-4e55-8087-97ece9671b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:37.552782 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1947b5-5815-4e55-8087-97ece9671b4c admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:37.553382 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1947b5-5815-4e55-8087-97ece9671b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:37.553610 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1947b5-5815-4e55-8087-97ece9671b4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:37.554536 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1947b5-5815-4e55-8087-97ece9671b4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:37.560279 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1947b5-5815-4e55-8087-97ece9671b4c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:37.634440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 843/1682] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:37 2022] POST /identity/v3/projects => generated 387 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:37.640722 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1401fa-b9eb-41d2-8a99-061c7d52b421 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:37.659687 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1401fa-b9eb-41d2-8a99-061c7d52b421 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:37.762315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:37.858698 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc9fa8-d47e-4ce1-a6d5-fb7208e53075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:37.865702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc9fa8-d47e-4ce1-a6d5-fb7208e53075 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:37.866716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc9fa8-d47e-4ce1-a6d5-fb7208e53075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:37.867214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc9fa8-d47e-4ce1-a6d5-fb7208e53075 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:37.868093 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc9fa8-d47e-4ce1-a6d5-fb7208e53075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:37.870988 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc9fa8-d47e-4ce1-a6d5-fb7208e53075 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:37.945095 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 840/1683] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:37 2022] POST /identity/v3/users => generated 300 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:37.946025 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1947b5-5815-4e55-8087-97ece9671b4c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:37.962263 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1947b5-5815-4e55-8087-97ece9671b4c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:37.986533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74d649-832b-466d-88db-aed959aeb47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:37.987382 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74d649-832b-466d-88db-aed959aeb47a admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:37.987834 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74d649-832b-466d-88db-aed959aeb47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:37.988254 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74d649-832b-466d-88db-aed959aeb47a admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:37.989075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74d649-832b-466d-88db-aed959aeb47a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:37.991877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74d649-832b-466d-88db-aed959aeb47a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:37.996287 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:38.020936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 844/1684] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:38.025293 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc9fa8-d47e-4ce1-a6d5-fb7208e53075 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:38.042702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc9fa8-d47e-4ce1-a6d5-fb7208e53075 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:38.061938 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87948cfc-17c1-4479-a5d2-fbdd47ac81de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:38.062444 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87948cfc-17c1-4479-a5d2-fbdd47ac81de admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:38.062717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87948cfc-17c1-4479-a5d2-fbdd47ac81de admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:38.062993 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87948cfc-17c1-4479-a5d2-fbdd47ac81de admin admin] PATH_INFO: `/v3/projects/e4a81211c97b4433a63d582100871e6a/users/5cf7a798d1d44e36bb0fe0db1ccf9a8a/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:38.093846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87948cfc-17c1-4479-a5d2-fbdd47ac81de admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4a81211c97b4433a63d582100871e6a, user_id=5cf7a798d1d44e36bb0fe0db1ccf9a8a, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:38.097282 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87948cfc-17c1-4479-a5d2-fbdd47ac81de admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:38.152988 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 841/1685] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:38 2022] PUT /identity/v3/projects/e4a81211c97b4433a63d582100871e6a/users/5cf7a798d1d44e36bb0fe0db1ccf9a8a/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:38.157230 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74d649-832b-466d-88db-aed959aeb47a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:38.171768 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74d649-832b-466d-88db-aed959aeb47a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:38.322848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b76d9-6683-453b-8720-f8a951dc9dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:38.324401 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b76d9-6683-453b-8720-f8a951dc9dfd admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:38.324973 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b76d9-6683-453b-8720-f8a951dc9dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:38.325391 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b76d9-6683-453b-8720-f8a951dc9dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:38.326191 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b76d9-6683-453b-8720-f8a951dc9dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:38.327993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b76d9-6683-453b-8720-f8a951dc9dfd admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:38.362181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87948cfc-17c1-4479-a5d2-fbdd47ac81de admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:38.363063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 845/1686] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:38.375595 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87948cfc-17c1-4479-a5d2-fbdd47ac81de admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:38.388209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229abf7b-93d7-42aa-94ae-751356430d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:38.390006 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229abf7b-93d7-42aa-94ae-751356430d6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:38.391120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229abf7b-93d7-42aa-94ae-751356430d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:38.391406 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229abf7b-93d7-42aa-94ae-751356430d6c admin admin] PATH_INFO: `/v3/projects/e4a81211c97b4433a63d582100871e6a/users/5cf7a798d1d44e36bb0fe0db1ccf9a8a/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:38.396790 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229abf7b-93d7-42aa-94ae-751356430d6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4a81211c97b4433a63d582100871e6a, user_id=5cf7a798d1d44e36bb0fe0db1ccf9a8a, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:38.413409 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229abf7b-93d7-42aa-94ae-751356430d6c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:38.476073 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e634b68-b7aa-471f-a1aa-96a18068cd63 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:38.476659 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 842/1687] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:38 2022] PUT /identity/v3/projects/e4a81211c97b4433a63d582100871e6a/users/5cf7a798d1d44e36bb0fe0db1ccf9a8a/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:38.476794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e634b68-b7aa-471f-a1aa-96a18068cd63 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:38.476938 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e634b68-b7aa-471f-a1aa-96a18068cd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:38.615259 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7e634b68-b7aa-471f-a1aa-96a18068cd63 None None] MFA Rules not processed for user `5cf7a798d1d44e36bb0fe0db1ccf9a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:38.616810 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7e634b68-b7aa-471f-a1aa-96a18068cd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:38.686639 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229abf7b-93d7-42aa-94ae-751356430d6c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:38.689343 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-229abf7b-93d7-42aa-94ae-751356430d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:38.813782 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 846/1688] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:38.827935 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e634b68-b7aa-471f-a1aa-96a18068cd63 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:38.831527 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7e634b68-b7aa-471f-a1aa-96a18068cd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:38.882774 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229abf7b-93d7-42aa-94ae-751356430d6c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:38.908292 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b007037-42df-49ed-a5f3-64de4836d219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:38.909237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b007037-42df-49ed-a5f3-64de4836d219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:38.909518 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b007037-42df-49ed-a5f3-64de4836d219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:38.909797 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b007037-42df-49ed-a5f3-64de4836d219 admin admin] PATH_INFO: `/v3/users/a1bb3995aade4889b7aba475f1d6d1d3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:38.911949 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b007037-42df-49ed-a5f3-64de4836d219 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1bb3995aade4889b7aba475f1d6d1d3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:38.913923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b007037-42df-49ed-a5f3-64de4836d219 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:38.998863 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e634b68-b7aa-471f-a1aa-96a18068cd63 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:39.009636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16c81f-bfd4-49c0-9a68-7ac88adfea6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:39.011171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16c81f-bfd4-49c0-9a68-7ac88adfea6e service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:39.012359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16c81f-bfd4-49c0-9a68-7ac88adfea6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:39.013297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16c81f-bfd4-49c0-9a68-7ac88adfea6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:39.040505 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16c81f-bfd4-49c0-9a68-7ac88adfea6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:39.045608 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16c81f-bfd4-49c0-9a68-7ac88adfea6e service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:39.051005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3b007037-42df-49ed-a5f3-64de4836d219 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1bb3995aade4889b7aba475f1d6d1d3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:39.064933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 847/1689] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:25:39.073468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3b007037-42df-49ed-a5f3-64de4836d219 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1bb3995aade4889b7aba475f1d6d1d3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:39.095293 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3b007037-42df-49ed-a5f3-64de4836d219 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1bb3995aade4889b7aba475f1d6d1d3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:39.096462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3b007037-42df-49ed-a5f3-64de4836d219 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1bb3995aade4889b7aba475f1d6d1d3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:39.099130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3b007037-42df-49ed-a5f3-64de4836d219 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1bb3995aade4889b7aba475f1d6d1d3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:39.179513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 843/1690] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:38 2022] DELETE /identity/v3/users/a1bb3995aade4889b7aba475f1d6d1d3 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:39.185016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc432eb2-3223-4a61-aafe-c9806dd15992 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:39.185567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc432eb2-3223-4a61-aafe-c9806dd15992 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:39.185915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc432eb2-3223-4a61-aafe-c9806dd15992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:39.229858 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b007037-42df-49ed-a5f3-64de4836d219 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:39.234370 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3b007037-42df-49ed-a5f3-64de4836d219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:39.240912 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-cc432eb2-3223-4a61-aafe-c9806dd15992 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:39.243040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cc432eb2-3223-4a61-aafe-c9806dd15992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:39.243938 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:39.293511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:39.406327 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 848/1691] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:39.433690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b007037-42df-49ed-a5f3-64de4836d219 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:39.451954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422df526-08ef-48eb-87c8-726127370083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:39.452853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422df526-08ef-48eb-87c8-726127370083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:39.453280 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422df526-08ef-48eb-87c8-726127370083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:39.453799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422df526-08ef-48eb-87c8-726127370083 admin admin] PATH_INFO: `/v3/users/e69ec152e64b4447a314a3db5c047412` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:39.455225 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422df526-08ef-48eb-87c8-726127370083 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e69ec152e64b4447a314a3db5c047412)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:39.457938 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422df526-08ef-48eb-87c8-726127370083 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:39.485104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc432eb2-3223-4a61-aafe-c9806dd15992 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:39.486817 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cc432eb2-3223-4a61-aafe-c9806dd15992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:39.507036 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc432eb2-3223-4a61-aafe-c9806dd15992 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:39.525246 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41734537-2dd6-46db-9abd-f605e394d2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:39.526085 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41734537-2dd6-46db-9abd-f605e394d2d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:39.526449 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41734537-2dd6-46db-9abd-f605e394d2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:39.528907 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41734537-2dd6-46db-9abd-f605e394d2d1 admin admin] PATH_INFO: `/v3/projects/bc649f835360455ba6e67121cd31586e` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:39.530499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41734537-2dd6-46db-9abd-f605e394d2d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc649f835360455ba6e67121cd31586e)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:39.532536 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41734537-2dd6-46db-9abd-f605e394d2d1 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:39.554733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-422df526-08ef-48eb-87c8-726127370083 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e69ec152e64b4447a314a3db5c047412'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:39.578280 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-422df526-08ef-48eb-87c8-726127370083 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e69ec152e64b4447a314a3db5c047412'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:39.587134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-422df526-08ef-48eb-87c8-726127370083 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e69ec152e64b4447a314a3db5c047412'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:39.589722 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-422df526-08ef-48eb-87c8-726127370083 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e69ec152e64b4447a314a3db5c047412'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:39.590478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-422df526-08ef-48eb-87c8-726127370083 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e69ec152e64b4447a314a3db5c047412'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:39.645135 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 844/1692] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:39 2022] DELETE /identity/v3/users/e69ec152e64b4447a314a3db5c047412 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:39.646377 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422df526-08ef-48eb-87c8-726127370083 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:39.654272 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-422df526-08ef-48eb-87c8-726127370083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:39.662441 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-41734537-2dd6-46db-9abd-f605e394d2d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc649f835360455ba6e67121cd31586e', 'request_id': 'req-41734537-2dd6-46db-9abd-f605e394d2d1'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:39.671452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-41734537-2dd6-46db-9abd-f605e394d2d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc649f835360455ba6e67121cd31586e', 'request_id': 'req-41734537-2dd6-46db-9abd-f605e394d2d1'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:39.703101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-41734537-2dd6-46db-9abd-f605e394d2d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc649f835360455ba6e67121cd31586e', 'request_id': 'req-41734537-2dd6-46db-9abd-f605e394d2d1'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:39.728895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-41734537-2dd6-46db-9abd-f605e394d2d1 admin admin] The token cache is being invalidate because project bc649f835360455ba6e67121cd31586e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:39.730426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-41734537-2dd6-46db-9abd-f605e394d2d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:39.731597 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-41734537-2dd6-46db-9abd-f605e394d2d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:39.738102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 849/1693] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:39 2022] DELETE /identity/v3/projects/bc649f835360455ba6e67121cd31586e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:25:39.739480 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba2878-3212-4226-82af-8990af033002 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:39.739809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba2878-3212-4226-82af-8990af033002 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:39.740041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba2878-3212-4226-82af-8990af033002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:39.786430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422df526-08ef-48eb-87c8-726127370083 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:39.792272 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af2c78-c780-49f5-aa55-a8f722bd9fd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:39.792776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af2c78-c780-49f5-aa55-a8f722bd9fd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:39.793054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af2c78-c780-49f5-aa55-a8f722bd9fd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:39.793510 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af2c78-c780-49f5-aa55-a8f722bd9fd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:39.795543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c4af2c78-c780-49f5-aa55-a8f722bd9fd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:39.808061 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-aeba2878-3212-4226-82af-8990af033002 None None] MFA Rules not processed for user `1fd1b40cbd884915bdfac150f9718e32`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:39.810495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-aeba2878-3212-4226-82af-8990af033002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:39.850389 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:39.893747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af2c78-c780-49f5-aa55-a8f722bd9fd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:39.895600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af2c78-c780-49f5-aa55-a8f722bd9fd6 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:39.911606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 850/1694] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:39.917124 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a490d-880e-4100-b156-796d680fe4e0 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:39.917447 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a490d-880e-4100-b156-796d680fe4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:39.917730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a490d-880e-4100-b156-796d680fe4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:39.922568 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 845/1695] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:39.923887 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaee662-2880-475e-a795-caac19191a2f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:39.924173 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaee662-2880-475e-a795-caac19191a2f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:39.924449 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaee662-2880-475e-a795-caac19191a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:39.960607 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a86a490d-880e-4100-b156-796d680fe4e0 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:39.961786 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a86a490d-880e-4100-b156-796d680fe4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:39.961903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6eaee662-2880-475e-a795-caac19191a2f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:39.971961 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6eaee662-2880-475e-a795-caac19191a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:40.004126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 851/1696] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:40.004126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718de6b-32ed-4582-9878-879d976947de None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:40.004126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718de6b-32ed-4582-9878-879d976947de None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:40.004126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718de6b-32ed-4582-9878-879d976947de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:40.044780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 846/1697] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:40.046240 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39916b23-c225-409a-a58b-3c94a4e3d273 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:40.046387 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39916b23-c225-409a-a58b-3c94a4e3d273 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:40.046643 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39916b23-c225-409a-a58b-3c94a4e3d273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:40.077880 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8718de6b-32ed-4582-9878-879d976947de None None] MFA Rules not processed for user `46ecdd9d0eb84701a651a1c687ad4522`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:40.079421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8718de6b-32ed-4582-9878-879d976947de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:40.099818 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-39916b23-c225-409a-a58b-3c94a4e3d273 None None] MFA Rules not processed for user `1fd1b40cbd884915bdfac150f9718e32`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:40.101079 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-39916b23-c225-409a-a58b-3c94a4e3d273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:40.105335 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:40.123130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 847/1698] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:40.128609 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39916b23-c225-409a-a58b-3c94a4e3d273 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:40.139594 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-39916b23-c225-409a-a58b-3c94a4e3d273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:40.172381 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39916b23-c225-409a-a58b-3c94a4e3d273 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:40.176787 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a9740-6be4-4320-9c25-6245d78b39c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:40.177706 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a9740-6be4-4320-9c25-6245d78b39c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:40.178334 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a9740-6be4-4320-9c25-6245d78b39c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:40.178746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a9740-6be4-4320-9c25-6245d78b39c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:40.199273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a9740-6be4-4320-9c25-6245d78b39c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:40.206171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 852/1699] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:40 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:40.206171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb85c28-5542-48e9-a65b-64ebcbc9db13 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:40.206171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb85c28-5542-48e9-a65b-64ebcbc9db13 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:40.206171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb85c28-5542-48e9-a65b-64ebcbc9db13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:40.208064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a9740-6be4-4320-9c25-6245d78b39c7 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:40.221131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 848/1700] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:40.222131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a9740-6be4-4320-9c25-6245d78b39c7 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:40.223689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-640a9740-6be4-4320-9c25-6245d78b39c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:40.282708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-feb85c28-5542-48e9-a65b-64ebcbc9db13 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:40.292370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-feb85c28-5542-48e9-a65b-64ebcbc9db13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:40.313713 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 853/1701] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:40.318079 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb85c28-5542-48e9-a65b-64ebcbc9db13 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:40.324131 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-feb85c28-5542-48e9-a65b-64ebcbc9db13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:40.357152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb85c28-5542-48e9-a65b-64ebcbc9db13 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:40.371971 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af623516-6134-4c6f-96f9-23a94345d4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:40.372764 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af623516-6134-4c6f-96f9-23a94345d4e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:40.373133 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af623516-6134-4c6f-96f9-23a94345d4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:40.373651 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af623516-6134-4c6f-96f9-23a94345d4e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:40.375037 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af623516-6134-4c6f-96f9-23a94345d4e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:40.377993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af623516-6134-4c6f-96f9-23a94345d4e0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:40.418199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a9740-6be4-4320-9c25-6245d78b39c7 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:40.434596 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915fe94-844c-440c-888b-75828060c4ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:40.437858 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915fe94-844c-440c-888b-75828060c4ba service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:40.444125 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915fe94-844c-440c-888b-75828060c4ba service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:40.445739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915fe94-844c-440c-888b-75828060c4ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:40.459810 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 854/1702] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:40 2022] POST /identity/v3/projects => generated 403 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:40.460709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af623516-6134-4c6f-96f9-23a94345d4e0 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:40.472313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915fe94-844c-440c-888b-75828060c4ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:40.478236 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915fe94-844c-440c-888b-75828060c4ba service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:40.508414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 849/1703] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:40 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:40.509427 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915fe94-844c-440c-888b-75828060c4ba service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:40.534948 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915fe94-844c-440c-888b-75828060c4ba service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:40.642230 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af623516-6134-4c6f-96f9-23a94345d4e0 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:40.657585 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9cbef-72f9-4f89-b678-6e284089d262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:40.658724 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9cbef-72f9-4f89-b678-6e284089d262 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:40.660733 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9cbef-72f9-4f89-b678-6e284089d262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:40.661701 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9cbef-72f9-4f89-b678-6e284089d262 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:40.662987 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df9cbef-72f9-4f89-b678-6e284089d262 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:40.663982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df9cbef-72f9-4f89-b678-6e284089d262 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:40.665753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df9cbef-72f9-4f89-b678-6e284089d262 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:40.703002 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 855/1704] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:40.706552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9cbef-72f9-4f89-b678-6e284089d262 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:40.711559 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171c3d1-28f6-406e-831d-07e7341782d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:40.712420 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171c3d1-28f6-406e-831d-07e7341782d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:40.713705 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171c3d1-28f6-406e-831d-07e7341782d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:40.716761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171c3d1-28f6-406e-831d-07e7341782d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:40.717663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171c3d1-28f6-406e-831d-07e7341782d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:40.724506 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171c3d1-28f6-406e-831d-07e7341782d1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:40.739002 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9cbef-72f9-4f89-b678-6e284089d262 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:40.758130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc511541-ecf6-40a4-82fd-9440495f5d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:40.759176 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc511541-ecf6-40a4-82fd-9440495f5d45 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:40.759712 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc511541-ecf6-40a4-82fd-9440495f5d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:40.762882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc511541-ecf6-40a4-82fd-9440495f5d45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:40.765234 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc511541-ecf6-40a4-82fd-9440495f5d45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:40.769020 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc511541-ecf6-40a4-82fd-9440495f5d45 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:40.787900 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 850/1705] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:40 2022] POST /identity/v3/users => generated 308 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:40.789806 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171c3d1-28f6-406e-831d-07e7341782d1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:40.824027 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171c3d1-28f6-406e-831d-07e7341782d1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:40.837279 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210c959-8e0a-4d22-9226-ae07214a16c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:40.841333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210c959-8e0a-4d22-9226-ae07214a16c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:40.845593 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210c959-8e0a-4d22-9226-ae07214a16c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:40.847407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210c959-8e0a-4d22-9226-ae07214a16c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:40.851266 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210c959-8e0a-4d22-9226-ae07214a16c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:40.855020 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 856/1706] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:40 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:40.855697 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210c959-8e0a-4d22-9226-ae07214a16c1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:40.859466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc511541-ecf6-40a4-82fd-9440495f5d45 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:40.883633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 851/1707] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:40.888314 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210c959-8e0a-4d22-9226-ae07214a16c1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:40.911926 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210c959-8e0a-4d22-9226-ae07214a16c1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:40.926296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:41.026913 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc511541-ecf6-40a4-82fd-9440495f5d45 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:41.040798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c2f53-9f17-4d0e-be4d-3a267d049853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:41.044020 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c2f53-9f17-4d0e-be4d-3a267d049853 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:41.044020 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c2f53-9f17-4d0e-be4d-3a267d049853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:41.044268 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c2f53-9f17-4d0e-be4d-3a267d049853 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:41.044917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c2f53-9f17-4d0e-be4d-3a267d049853 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:41.048630 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c2f53-9f17-4d0e-be4d-3a267d049853 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:41.084350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480d89e-bdde-4d14-80d0-cdecf4d5bd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:41.085131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480d89e-bdde-4d14-80d0-cdecf4d5bd52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:41.085418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480d89e-bdde-4d14-80d0-cdecf4d5bd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:41.085694 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480d89e-bdde-4d14-80d0-cdecf4d5bd52 admin admin] PATH_INFO: `/v3/projects/04b5b49139c246a1a8e0c8107c8bf4ca/users/a4bd5e92711b4737be207f3b235a7992/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:41.092196 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:41.129962 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 857/1708] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:40 2022] POST /identity/v3/users => generated 288 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:41.133552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c2f53-9f17-4d0e-be4d-3a267d049853 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:41.148178 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480d89e-bdde-4d14-80d0-cdecf4d5bd52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04b5b49139c246a1a8e0c8107c8bf4ca, user_id=a4bd5e92711b4737be207f3b235a7992, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:41.154313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480d89e-bdde-4d14-80d0-cdecf4d5bd52 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:41.163654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c2f53-9f17-4d0e-be4d-3a267d049853 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:41.174971 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ae216-7c00-4304-81a4-f8fbad372dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:41.176168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ae216-7c00-4304-81a4-f8fbad372dec admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:41.176655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ae216-7c00-4304-81a4-f8fbad372dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:41.187903 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ae216-7c00-4304-81a4-f8fbad372dec admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:41.188640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ae216-7c00-4304-81a4-f8fbad372dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:41.192871 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 852/1709] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:40 2022] PUT /identity/v3/projects/04b5b49139c246a1a8e0c8107c8bf4ca/users/a4bd5e92711b4737be207f3b235a7992/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:41.193035 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ae216-7c00-4304-81a4-f8fbad372dec admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:41.194561 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480d89e-bdde-4d14-80d0-cdecf4d5bd52 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:41.229612 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 858/1710] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:41.231020 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ae216-7c00-4304-81a4-f8fbad372dec admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:41.255495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ae216-7c00-4304-81a4-f8fbad372dec admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:41.377728 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480d89e-bdde-4d14-80d0-cdecf4d5bd52 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:41.391162 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2379a7e-d035-4856-87e2-c5babb336103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:41.392159 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2379a7e-d035-4856-87e2-c5babb336103 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:41.392638 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2379a7e-d035-4856-87e2-c5babb336103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:41.392919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2379a7e-d035-4856-87e2-c5babb336103 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:41.393630 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2379a7e-d035-4856-87e2-c5babb336103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:41.395253 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2379a7e-d035-4856-87e2-c5babb336103 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:41.424395 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0648bb8-4e05-4265-be58-99e376f42bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:41.425172 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0648bb8-4e05-4265-be58-99e376f42bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:41.425276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0648bb8-4e05-4265-be58-99e376f42bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:41.426629 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0648bb8-4e05-4265-be58-99e376f42bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:41.429929 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 853/1711] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:41.430012 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0648bb8-4e05-4265-be58-99e376f42bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:41.430012 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0648bb8-4e05-4265-be58-99e376f42bd7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:41.431235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2379a7e-d035-4856-87e2-c5babb336103 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:41.444911 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2379a7e-d035-4856-87e2-c5babb336103 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:41.460190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 859/1712] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:41.464339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d89c9-1a44-490b-8f50-a4afc53a2404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:41.464713 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0648bb8-4e05-4265-be58-99e376f42bd7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:41.465029 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d89c9-1a44-490b-8f50-a4afc53a2404 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:41.465320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d89c9-1a44-490b-8f50-a4afc53a2404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:41.465561 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d89c9-1a44-490b-8f50-a4afc53a2404 admin admin] PATH_INFO: `/v3/projects/04b5b49139c246a1a8e0c8107c8bf4ca/users/a4bd5e92711b4737be207f3b235a7992/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:41.468124 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d89c9-1a44-490b-8f50-a4afc53a2404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04b5b49139c246a1a8e0c8107c8bf4ca, user_id=a4bd5e92711b4737be207f3b235a7992, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:41.470920 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d89c9-1a44-490b-8f50-a4afc53a2404 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:41.483439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0648bb8-4e05-4265-be58-99e376f42bd7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:41.496678 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3990c41-f352-40bf-ba42-dd5fea3526c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:41.497343 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3990c41-f352-40bf-ba42-dd5fea3526c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:41.499141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3990c41-f352-40bf-ba42-dd5fea3526c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:41.499439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3990c41-f352-40bf-ba42-dd5fea3526c1 admin admin] PATH_INFO: `/v3/projects/7abac19f224a481ea5cae1008061f6b8/users/883d1b23657f4c22a7d34ffdda49215f/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:41.514552 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 854/1713] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:41 2022] PUT /identity/v3/projects/04b5b49139c246a1a8e0c8107c8bf4ca/users/a4bd5e92711b4737be207f3b235a7992/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:25:41.529981 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3990c41-f352-40bf-ba42-dd5fea3526c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7abac19f224a481ea5cae1008061f6b8, user_id=883d1b23657f4c22a7d34ffdda49215f, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:41.532829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3990c41-f352-40bf-ba42-dd5fea3526c1 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:41.554081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 860/1714] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:41 2022] PUT /identity/v3/projects/7abac19f224a481ea5cae1008061f6b8/users/883d1b23657f4c22a7d34ffdda49215f/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:41.564633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c33d93-3a02-45ef-ad25-c0d4058fb05f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:41.565040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c33d93-3a02-45ef-ad25-c0d4058fb05f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:41.565452 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c33d93-3a02-45ef-ad25-c0d4058fb05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:41.628446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-21c33d93-3a02-45ef-ad25-c0d4058fb05f None None] MFA Rules not processed for user `883d1b23657f4c22a7d34ffdda49215f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:41.632901 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-21c33d93-3a02-45ef-ad25-c0d4058fb05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:41.737297 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 855/1715] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:25:41.759591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3990c41-f352-40bf-ba42-dd5fea3526c1 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:41.762191 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e3990c41-f352-40bf-ba42-dd5fea3526c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:41.804417 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919e104-5f34-4e63-ac26-6fb5cd5cd5fe None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:41.804702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919e104-5f34-4e63-ac26-6fb5cd5cd5fe None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:41.804972 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919e104-5f34-4e63-ac26-6fb5cd5cd5fe None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:41.806672 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 856/1716] 10.222.0.39 () {54 vars in 912 bytes} [Mon Aug 8 02:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:25:41.812387 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7c468-e584-459a-98c9-fde88314a45b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:41.812715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7c468-e584-459a-98c9-fde88314a45b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:41.812937 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7c468-e584-459a-98c9-fde88314a45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:41.824705 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:41.907591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3990c41-f352-40bf-ba42-dd5fea3526c1 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:41.908409 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e0f7c468-e584-459a-98c9-fde88314a45b None None] MFA Rules not processed for user `78506fea1c024b169a4b7ee86116658c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:41.913154 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e0f7c468-e584-459a-98c9-fde88314a45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:41.918098 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23c303-24f5-4ef7-863c-436c7efe8480 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:41.919811 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23c303-24f5-4ef7-863c-436c7efe8480 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:41.920377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23c303-24f5-4ef7-863c-436c7efe8480 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:41.920985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23c303-24f5-4ef7-863c-436c7efe8480 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:41.952066 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23c303-24f5-4ef7-863c-436c7efe8480 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:41.955538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23c303-24f5-4ef7-863c-436c7efe8480 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:41.987066 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 861/1717] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:42.070154 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 857/1718] 10.222.0.39 () {58 vars in 1009 bytes} [Mon Aug 8 02:25:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:25:42.085606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23c303-24f5-4ef7-863c-436c7efe8480 service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:42.087862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9e23c303-24f5-4ef7-863c-436c7efe8480 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:42.119909 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7c468-e584-459a-98c9-fde88314a45b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:42.145995 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7c468-e584-459a-98c9-fde88314a45b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:42.172461 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f714419-d6e2-4937-bf42-4229c2f41ce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:42.173170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f714419-d6e2-4937-bf42-4229c2f41ce9 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:42.173362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f714419-d6e2-4937-bf42-4229c2f41ce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:42.173634 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f714419-d6e2-4937-bf42-4229c2f41ce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:42.196982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:42.206911 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f714419-d6e2-4937-bf42-4229c2f41ce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:42.211659 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f714419-d6e2-4937-bf42-4229c2f41ce9 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:42.241782 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23c303-24f5-4ef7-863c-436c7efe8480 service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:42.249163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e4129-dc9f-4c82-b67b-5a575ec32e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:42.250059 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e4129-dc9f-4c82-b67b-5a575ec32e3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:42.250390 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e4129-dc9f-4c82-b67b-5a575ec32e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:42.250740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e4129-dc9f-4c82-b67b-5a575ec32e3f admin admin] PATH_INFO: `/v3/projects/f673fd6851904ccbb5a916a8bdea0ab9` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:42.252212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e4129-dc9f-4c82-b67b-5a575ec32e3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f673fd6851904ccbb5a916a8bdea0ab9)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:42.254495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e4129-dc9f-4c82-b67b-5a575ec32e3f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:42.261147 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 858/1719] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:42.262068 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f714419-d6e2-4937-bf42-4229c2f41ce9 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:42.271066 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9f714419-d6e2-4937-bf42-4229c2f41ce9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:42.316955 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f714419-d6e2-4937-bf42-4229c2f41ce9 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:42.333935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda5bed-f7c5-41a7-908a-728ba9712989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:42.334654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda5bed-f7c5-41a7-908a-728ba9712989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:42.334930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda5bed-f7c5-41a7-908a-728ba9712989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:42.335904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda5bed-f7c5-41a7-908a-728ba9712989 admin admin] PATH_INFO: `/v3/projects/911b7361e9064f0eafe6fa1328042556` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:42.338784 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda5bed-f7c5-41a7-908a-728ba9712989 admin admin] RBAC: Authorizing `identity:delete_project(project_id=911b7361e9064f0eafe6fa1328042556)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:42.341354 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda5bed-f7c5-41a7-908a-728ba9712989 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:42.396086 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-623e4129-dc9f-4c82-b67b-5a575ec32e3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f673fd6851904ccbb5a916a8bdea0ab9', 'request_id': 'req-623e4129-dc9f-4c82-b67b-5a575ec32e3f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.411956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-623e4129-dc9f-4c82-b67b-5a575ec32e3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f673fd6851904ccbb5a916a8bdea0ab9', 'request_id': 'req-623e4129-dc9f-4c82-b67b-5a575ec32e3f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.437164 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-623e4129-dc9f-4c82-b67b-5a575ec32e3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f673fd6851904ccbb5a916a8bdea0ab9', 'request_id': 'req-623e4129-dc9f-4c82-b67b-5a575ec32e3f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.442708 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9bda5bed-f7c5-41a7-908a-728ba9712989 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '911b7361e9064f0eafe6fa1328042556', 'request_id': 'req-9bda5bed-f7c5-41a7-908a-728ba9712989'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.456703 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-623e4129-dc9f-4c82-b67b-5a575ec32e3f admin admin] The token cache is being invalidate because project f673fd6851904ccbb5a916a8bdea0ab9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:42.457001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-623e4129-dc9f-4c82-b67b-5a575ec32e3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.458688 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-623e4129-dc9f-4c82-b67b-5a575ec32e3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.459110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9bda5bed-f7c5-41a7-908a-728ba9712989 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '911b7361e9064f0eafe6fa1328042556', 'request_id': 'req-9bda5bed-f7c5-41a7-908a-728ba9712989'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.471032 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 862/1720] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:42 2022] DELETE /identity/v3/projects/f673fd6851904ccbb5a916a8bdea0ab9 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:42.471032 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e4129-dc9f-4c82-b67b-5a575ec32e3f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:42.471032 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-623e4129-dc9f-4c82-b67b-5a575ec32e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:42.474178 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9bda5bed-f7c5-41a7-908a-728ba9712989 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '911b7361e9064f0eafe6fa1328042556', 'request_id': 'req-9bda5bed-f7c5-41a7-908a-728ba9712989'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.491445 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9bda5bed-f7c5-41a7-908a-728ba9712989 admin admin] The token cache is being invalidate because project 911b7361e9064f0eafe6fa1328042556 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:42.492456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9bda5bed-f7c5-41a7-908a-728ba9712989 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.492825 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9bda5bed-f7c5-41a7-908a-728ba9712989 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.495671 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 859/1721] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:42 2022] DELETE /identity/v3/projects/911b7361e9064f0eafe6fa1328042556 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:42.497330 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4001ede-8cce-4eb6-b9ad-17b638afaae2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:42.497724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4001ede-8cce-4eb6-b9ad-17b638afaae2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:42.498153 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4001ede-8cce-4eb6-b9ad-17b638afaae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:42.525126 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d4001ede-8cce-4eb6-b9ad-17b638afaae2 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:42.527208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d4001ede-8cce-4eb6-b9ad-17b638afaae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:42.607262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e4129-dc9f-4c82-b67b-5a575ec32e3f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:42.612277 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56623574-42ab-41ab-9e32-baaf1b041157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:42.613119 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56623574-42ab-41ab-9e32-baaf1b041157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:42.613236 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56623574-42ab-41ab-9e32-baaf1b041157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:42.613466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56623574-42ab-41ab-9e32-baaf1b041157 admin admin] PATH_INFO: `/v3/users/883d1b23657f4c22a7d34ffdda49215f` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:42.621524 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 860/1722] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:42.621612 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56623574-42ab-41ab-9e32-baaf1b041157 admin admin] RBAC: Authorizing `identity:delete_user(user_id=883d1b23657f4c22a7d34ffdda49215f)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:42.621682 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4001ede-8cce-4eb6-b9ad-17b638afaae2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:42.621747 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56623574-42ab-41ab-9e32-baaf1b041157 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:42.621852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d4001ede-8cce-4eb6-b9ad-17b638afaae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:42.652569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4001ede-8cce-4eb6-b9ad-17b638afaae2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:42.659867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb0db3-3237-4c1f-8eba-ae1f8d437445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:42.665656 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb0db3-3237-4c1f-8eba-ae1f8d437445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:42.665955 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb0db3-3237-4c1f-8eba-ae1f8d437445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:42.667000 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb0db3-3237-4c1f-8eba-ae1f8d437445 admin admin] PATH_INFO: `/v3/users/5cf7a798d1d44e36bb0fe0db1ccf9a8a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:42.668714 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb0db3-3237-4c1f-8eba-ae1f8d437445 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cf7a798d1d44e36bb0fe0db1ccf9a8a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:42.671123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb0db3-3237-4c1f-8eba-ae1f8d437445 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:42.693099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:42.705915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-56623574-42ab-41ab-9e32-baaf1b041157 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '883d1b23657f4c22a7d34ffdda49215f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.721713 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-56623574-42ab-41ab-9e32-baaf1b041157 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '883d1b23657f4c22a7d34ffdda49215f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.743439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-56623574-42ab-41ab-9e32-baaf1b041157 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '883d1b23657f4c22a7d34ffdda49215f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.744353 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-56623574-42ab-41ab-9e32-baaf1b041157 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '883d1b23657f4c22a7d34ffdda49215f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.745437 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-56623574-42ab-41ab-9e32-baaf1b041157 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '883d1b23657f4c22a7d34ffdda49215f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.757760 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:42.795409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 863/1723] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:42 2022] DELETE /identity/v3/users/883d1b23657f4c22a7d34ffdda49215f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:42.802585 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64a1d8-4b9f-4ecd-99b3-c903c0d55f02 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:42.803452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64a1d8-4b9f-4ecd-99b3-c903c0d55f02 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:42.804015 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-06bb0db3-3237-4c1f-8eba-ae1f8d437445 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cf7a798d1d44e36bb0fe0db1ccf9a8a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.804945 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64a1d8-4b9f-4ecd-99b3-c903c0d55f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:42.832603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-06bb0db3-3237-4c1f-8eba-ae1f8d437445 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cf7a798d1d44e36bb0fe0db1ccf9a8a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.833412 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ea64a1d8-4b9f-4ecd-99b3-c903c0d55f02 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:42.835137 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ea64a1d8-4b9f-4ecd-99b3-c903c0d55f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:42.845365 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-06bb0db3-3237-4c1f-8eba-ae1f8d437445 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cf7a798d1d44e36bb0fe0db1ccf9a8a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.848277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-06bb0db3-3237-4c1f-8eba-ae1f8d437445 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cf7a798d1d44e36bb0fe0db1ccf9a8a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.849430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-06bb0db3-3237-4c1f-8eba-ae1f8d437445 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cf7a798d1d44e36bb0fe0db1ccf9a8a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:42.893020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 861/1724] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:42 2022] DELETE /identity/v3/users/5cf7a798d1d44e36bb0fe0db1ccf9a8a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:42.894853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91237fc1-7b3e-449b-b6aa-e84296169b19 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:42.895311 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91237fc1-7b3e-449b-b6aa-e84296169b19 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:42.896057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91237fc1-7b3e-449b-b6aa-e84296169b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:42.914285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 864/1725] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:42.917315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64a1d8-4b9f-4ecd-99b3-c903c0d55f02 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:42.930771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64a1d8-4b9f-4ecd-99b3-c903c0d55f02 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:42.952562 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-91237fc1-7b3e-449b-b6aa-e84296169b19 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:42.954034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-91237fc1-7b3e-449b-b6aa-e84296169b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:43.006893 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb29574-1626-4e90-a0e3-4633db536217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:43.007492 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb29574-1626-4e90-a0e3-4633db536217 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:43.007833 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb29574-1626-4e90-a0e3-4633db536217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:43.008784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb29574-1626-4e90-a0e3-4633db536217 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:43.009462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb29574-1626-4e90-a0e3-4633db536217 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:43.010225 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb29574-1626-4e90-a0e3-4633db536217 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:43.011463 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb29574-1626-4e90-a0e3-4633db536217 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:43.028765 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 865/1726] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:43.033022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb29574-1626-4e90-a0e3-4633db536217 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:43.060677 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb29574-1626-4e90-a0e3-4633db536217 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:43.077238 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146186bf-6ecb-4519-98f1-43df80952d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:43.077875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146186bf-6ecb-4519-98f1-43df80952d13 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:43.078153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146186bf-6ecb-4519-98f1-43df80952d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:43.078455 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146186bf-6ecb-4519-98f1-43df80952d13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:43.079568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146186bf-6ecb-4519-98f1-43df80952d13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:43.080728 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 862/1727] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:43.086390 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146186bf-6ecb-4519-98f1-43df80952d13 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:43.145044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 866/1728] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:43 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:43.150348 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91237fc1-7b3e-449b-b6aa-e84296169b19 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:43.165564 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91237fc1-7b3e-449b-b6aa-e84296169b19 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:43.203141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146186bf-6ecb-4519-98f1-43df80952d13 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:43.205117 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-146186bf-6ecb-4519-98f1-43df80952d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:43.256406 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073dd43-cd37-4d7d-8ce3-4286575c827d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:43.258839 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073dd43-cd37-4d7d-8ce3-4286575c827d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:43.259930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073dd43-cd37-4d7d-8ce3-4286575c827d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:43.260589 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073dd43-cd37-4d7d-8ce3-4286575c827d admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:43.262152 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073dd43-cd37-4d7d-8ce3-4286575c827d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:43.268584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073dd43-cd37-4d7d-8ce3-4286575c827d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:43.322682 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 863/1729] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:43 2022] POST /identity/v3/users => generated 291 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:43.325561 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073dd43-cd37-4d7d-8ce3-4286575c827d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:43.352191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073dd43-cd37-4d7d-8ce3-4286575c827d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:43.359627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146186bf-6ecb-4519-98f1-43df80952d13 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:43.367641 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e348df-cc76-4319-868c-3338bdec724f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:43.369081 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e348df-cc76-4319-868c-3338bdec724f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:43.369517 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e348df-cc76-4319-868c-3338bdec724f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:43.370026 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ed8fb-5c1e-4c69-83a2-b9892a954aa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:43.370936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e348df-cc76-4319-868c-3338bdec724f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:43.372121 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ed8fb-5c1e-4c69-83a2-b9892a954aa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:43.372679 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e348df-cc76-4319-868c-3338bdec724f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:43.373189 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ed8fb-5c1e-4c69-83a2-b9892a954aa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:43.373929 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ed8fb-5c1e-4c69-83a2-b9892a954aa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:43.379108 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e348df-cc76-4319-868c-3338bdec724f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:43.399714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ed8fb-5c1e-4c69-83a2-b9892a954aa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:43.404190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ed8fb-5c1e-4c69-83a2-b9892a954aa5 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:43.412884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 864/1730] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:43.413123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e348df-cc76-4319-868c-3338bdec724f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:43.425353 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:43.435930 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 867/1731] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:43.438867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e348df-cc76-4319-868c-3338bdec724f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:43.452361 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabc581-6867-494c-96b2-95d3b5ee461d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:43.454072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabc581-6867-494c-96b2-95d3b5ee461d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:43.454414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabc581-6867-494c-96b2-95d3b5ee461d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:43.454695 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabc581-6867-494c-96b2-95d3b5ee461d admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:43.455770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabc581-6867-494c-96b2-95d3b5ee461d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:43.457105 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabc581-6867-494c-96b2-95d3b5ee461d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:43.476077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 865/1732] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:43.481489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ed8fb-5c1e-4c69-83a2-b9892a954aa5 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:43.493541 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:43.519636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ed8fb-5c1e-4c69-83a2-b9892a954aa5 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:43.534044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfd053-fcfd-4505-9bdc-21b795e0dda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:43.536170 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfd053-fcfd-4505-9bdc-21b795e0dda6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:43.538083 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfd053-fcfd-4505-9bdc-21b795e0dda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:43.539895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfd053-fcfd-4505-9bdc-21b795e0dda6 admin admin] PATH_INFO: `/v3/projects/a6ef7e48e4194a3992deb964c68af44c/users/cc2e26059372409e9f6e263942063e15/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:43.602566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfd053-fcfd-4505-9bdc-21b795e0dda6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ef7e48e4194a3992deb964c68af44c, user_id=cc2e26059372409e9f6e263942063e15, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:43.611193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfd053-fcfd-4505-9bdc-21b795e0dda6 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:43.655207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff6440-951c-4dcd-83bc-fc768569d24f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:43.655493 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff6440-951c-4dcd-83bc-fc768569d24f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:43.655993 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff6440-951c-4dcd-83bc-fc768569d24f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:43.657676 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 866/1734] 10.222.0.39 () {54 vars in 961 bytes} [Mon Aug 8 02:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:43.659578 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 868/1734] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:43 2022] PUT /identity/v3/projects/a6ef7e48e4194a3992deb964c68af44c/users/cc2e26059372409e9f6e263942063e15/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:43.660885 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dc26a-1e23-4e2c-b9ff-065c9c56e592 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:43.661169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dc26a-1e23-4e2c-b9ff-065c9c56e592 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:43.661438 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dc26a-1e23-4e2c-b9ff-065c9c56e592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:43.670397 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7718e65-271d-4f1b-9eb0-800aa4e9eda3 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:43.670973 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7718e65-271d-4f1b-9eb0-800aa4e9eda3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:43.671909 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7718e65-271d-4f1b-9eb0-800aa4e9eda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:43.723778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c7718e65-271d-4f1b-9eb0-800aa4e9eda3 None None] MFA Rules not processed for user `f62745b92a864480a9438de581e1d05e`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:43.723982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-195dc26a-1e23-4e2c-b9ff-065c9c56e592 None None] MFA Rules not processed for user `cc2e26059372409e9f6e263942063e15`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:43.731422 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c7718e65-271d-4f1b-9eb0-800aa4e9eda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:43.731766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-195dc26a-1e23-4e2c-b9ff-065c9c56e592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:43.971580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 869/1735] 10.222.0.39 () {58 vars in 1058 bytes} [Mon Aug 8 02:25:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:25:43.981208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7718e65-271d-4f1b-9eb0-800aa4e9eda3 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:43.996829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 867/1736] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:43.999020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195dc26a-1e23-4e2c-b9ff-065c9c56e592 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:44.004513 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7718e65-271d-4f1b-9eb0-800aa4e9eda3 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:44.023244 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fdd36-65fc-4171-b91e-d88e1a9a6b3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f62745b92a864480a9438de581e1d05e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:44.023876 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fdd36-65fc-4171-b91e-d88e1a9a6b3f service placement] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:44.024161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fdd36-65fc-4171-b91e-d88e1a9a6b3f service placement] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:44.024437 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fdd36-65fc-4171-b91e-d88e1a9a6b3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:44.027563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-430fdd36-65fc-4171-b91e-d88e1a9a6b3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:44.062307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fdd36-65fc-4171-b91e-d88e1a9a6b3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:44.064323 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fdd36-65fc-4171-b91e-d88e1a9a6b3f service placement] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:44.084023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 870/1737] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:25:44.134881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195dc26a-1e23-4e2c-b9ff-065c9c56e592 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:44.142522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568132ce-d575-454e-9ba4-1dea313be09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:44.143314 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568132ce-d575-454e-9ba4-1dea313be09e admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:44.143706 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568132ce-d575-454e-9ba4-1dea313be09e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:44.144002 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568132ce-d575-454e-9ba4-1dea313be09e admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:44.144707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568132ce-d575-454e-9ba4-1dea313be09e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:44.147404 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568132ce-d575-454e-9ba4-1dea313be09e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:44.213723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fdd36-65fc-4171-b91e-d88e1a9a6b3f service placement] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:44.214351 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 868/1738] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:43 2022] POST /identity/v3/projects => generated 379 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:44.231415 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fdd36-65fc-4171-b91e-d88e1a9a6b3f service placement] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:44.396234 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9c4bf-ee29-4385-8344-51cc2ff02f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:44.396840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9c4bf-ee29-4385-8344-51cc2ff02f5c admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:44.397126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9c4bf-ee29-4385-8344-51cc2ff02f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:44.397396 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9c4bf-ee29-4385-8344-51cc2ff02f5c admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:44.398026 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9c4bf-ee29-4385-8344-51cc2ff02f5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:44.399997 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9c4bf-ee29-4385-8344-51cc2ff02f5c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:44.413474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4295761-11ad-4022-8660-5bc5a2b55f7f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:44.413777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4295761-11ad-4022-8660-5bc5a2b55f7f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:44.414022 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4295761-11ad-4022-8660-5bc5a2b55f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:44.456518 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 871/1739] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:44 2022] POST /identity/v3/users => generated 296 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:44.457093 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9c4bf-ee29-4385-8344-51cc2ff02f5c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:44.472560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d4295761-11ad-4022-8660-5bc5a2b55f7f None None] MFA Rules not processed for user `1fd1b40cbd884915bdfac150f9718e32`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:44.474084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d4295761-11ad-4022-8660-5bc5a2b55f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:44.476955 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9c4bf-ee29-4385-8344-51cc2ff02f5c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:44.500571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78157990-3913-4784-a33c-5727c1b1995e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:44.501114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78157990-3913-4784-a33c-5727c1b1995e admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:44.501411 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78157990-3913-4784-a33c-5727c1b1995e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:44.501671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78157990-3913-4784-a33c-5727c1b1995e admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:44.502368 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78157990-3913-4784-a33c-5727c1b1995e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:44.503998 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78157990-3913-4784-a33c-5727c1b1995e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:44.532965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 872/1740] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:44.537242 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78157990-3913-4784-a33c-5727c1b1995e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:44.560489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78157990-3913-4784-a33c-5727c1b1995e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:44.592835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f8c3f-3fdf-4e00-91ba-c231d3ea7220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:44.593525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f8c3f-3fdf-4e00-91ba-c231d3ea7220 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:44.593809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f8c3f-3fdf-4e00-91ba-c231d3ea7220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:44.594086 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f8c3f-3fdf-4e00-91ba-c231d3ea7220 admin admin] PATH_INFO: `/v3/projects/63eed118baf74e0da41a241504d33c92/users/e1a11c3c58a34a019b86e5e58740fbf9/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:44.664520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f8c3f-3fdf-4e00-91ba-c231d3ea7220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63eed118baf74e0da41a241504d33c92, user_id=e1a11c3c58a34a019b86e5e58740fbf9, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:44.668213 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f8c3f-3fdf-4e00-91ba-c231d3ea7220 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:44.675846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 869/1741] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:44.695142 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4295761-11ad-4022-8660-5bc5a2b55f7f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:44.696017 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 873/1742] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:44 2022] PUT /identity/v3/projects/63eed118baf74e0da41a241504d33c92/users/e1a11c3c58a34a019b86e5e58740fbf9/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:25:44.711445 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4295761-11ad-4022-8660-5bc5a2b55f7f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:44.740442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:44.959231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6f1bf-3858-4f0a-bdc6-8d2b169de693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:44.961337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6f1bf-3858-4f0a-bdc6-8d2b169de693 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:44.961618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6f1bf-3858-4f0a-bdc6-8d2b169de693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:44.961888 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6f1bf-3858-4f0a-bdc6-8d2b169de693 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:44.962586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6f1bf-3858-4f0a-bdc6-8d2b169de693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:44.968286 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6f1bf-3858-4f0a-bdc6-8d2b169de693 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:44.983859 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b32af5-9fed-49dd-bfd1-d0e6ac2b5b5a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:44.984400 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b32af5-9fed-49dd-bfd1-d0e6ac2b5b5a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:44.984874 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b32af5-9fed-49dd-bfd1-d0e6ac2b5b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:44.990652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 870/1743] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:44.992831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6f1bf-3858-4f0a-bdc6-8d2b169de693 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:45.018842 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6f1bf-3858-4f0a-bdc6-8d2b169de693 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:45.032471 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697874ed-0a0e-48e8-8554-c8551ddbba7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:45.033141 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697874ed-0a0e-48e8-8554-c8551ddbba7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:45.033414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697874ed-0a0e-48e8-8554-c8551ddbba7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:45.033682 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697874ed-0a0e-48e8-8554-c8551ddbba7a admin admin] PATH_INFO: `/v3/projects/63eed118baf74e0da41a241504d33c92/users/e1a11c3c58a34a019b86e5e58740fbf9/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:45.038557 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697874ed-0a0e-48e8-8554-c8551ddbba7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=63eed118baf74e0da41a241504d33c92, user_id=e1a11c3c58a34a019b86e5e58740fbf9, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:45.043664 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697874ed-0a0e-48e8-8554-c8551ddbba7a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:45.044556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-63b32af5-9fed-49dd-bfd1-d0e6ac2b5b5a None None] MFA Rules not processed for user `43dd72b161f8442285dea411741d66b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:45.050343 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-63b32af5-9fed-49dd-bfd1-d0e6ac2b5b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:45.075407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 871/1744] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:44 2022] PUT /identity/v3/projects/63eed118baf74e0da41a241504d33c92/users/e1a11c3c58a34a019b86e5e58740fbf9/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:45.085916 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356676e9-9089-425c-aa1b-617e25679b48 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:45.086205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356676e9-9089-425c-aa1b-617e25679b48 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:45.092334 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356676e9-9089-425c-aa1b-617e25679b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:45.094480 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:45.178081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 874/1745] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:45.180473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430c105-1915-4947-a168-dea3b2b65672 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:45.181000 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430c105-1915-4947-a168-dea3b2b65672 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:45.181640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430c105-1915-4947-a168-dea3b2b65672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:45.184311 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-356676e9-9089-425c-aa1b-617e25679b48 None None] MFA Rules not processed for user `e1a11c3c58a34a019b86e5e58740fbf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:45.185571 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-356676e9-9089-425c-aa1b-617e25679b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:45.259244 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8430c105-1915-4947-a168-dea3b2b65672 None None] MFA Rules not processed for user `17286ff2e1764ec7b91e6274c7162437`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:45.264606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8430c105-1915-4947-a168-dea3b2b65672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:45.427856 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 872/1746] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:45.438519 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356676e9-9089-425c-aa1b-617e25679b48 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:45.445366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-356676e9-9089-425c-aa1b-617e25679b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:45.482388 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 875/1747] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:45.562933 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356676e9-9089-425c-aa1b-617e25679b48 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:45.573462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c26a43-2344-48f6-8903-792fd5f35d06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:45.574105 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c26a43-2344-48f6-8903-792fd5f35d06 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:45.575059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c26a43-2344-48f6-8903-792fd5f35d06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:45.575059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c26a43-2344-48f6-8903-792fd5f35d06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:45.600553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c26a43-2344-48f6-8903-792fd5f35d06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:45.602528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c26a43-2344-48f6-8903-792fd5f35d06 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:45.631045 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 873/1748] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:45.691140 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e096bd-0dde-4b95-bcc1-fefbb40a6925 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:45.691433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e096bd-0dde-4b95-bcc1-fefbb40a6925 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:45.691710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e096bd-0dde-4b95-bcc1-fefbb40a6925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:45.748170 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-66e096bd-0dde-4b95-bcc1-fefbb40a6925 None None] MFA Rules not processed for user `a4bd5e92711b4737be207f3b235a7992`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:45.749477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-66e096bd-0dde-4b95-bcc1-fefbb40a6925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:45.893941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c26a43-2344-48f6-8903-792fd5f35d06 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:45.895630 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b3c26a43-2344-48f6-8903-792fd5f35d06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:45.957185 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 876/1749] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:45 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:45.965529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:46.047813 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c26a43-2344-48f6-8903-792fd5f35d06 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:46.057993 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43743bf3-7662-48b3-85ee-c92e4ce73466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:46.059092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43743bf3-7662-48b3-85ee-c92e4ce73466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:46.059896 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43743bf3-7662-48b3-85ee-c92e4ce73466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:46.061558 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43743bf3-7662-48b3-85ee-c92e4ce73466 admin admin] PATH_INFO: `/v3/projects/e4a81211c97b4433a63d582100871e6a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:46.063207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43743bf3-7662-48b3-85ee-c92e4ce73466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4a81211c97b4433a63d582100871e6a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:46.066881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43743bf3-7662-48b3-85ee-c92e4ce73466 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:46.159494 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-43743bf3-7662-48b3-85ee-c92e4ce73466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4a81211c97b4433a63d582100871e6a', 'request_id': 'req-43743bf3-7662-48b3-85ee-c92e4ce73466'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:46.170896 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-43743bf3-7662-48b3-85ee-c92e4ce73466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4a81211c97b4433a63d582100871e6a', 'request_id': 'req-43743bf3-7662-48b3-85ee-c92e4ce73466'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:46.197728 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-43743bf3-7662-48b3-85ee-c92e4ce73466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4a81211c97b4433a63d582100871e6a', 'request_id': 'req-43743bf3-7662-48b3-85ee-c92e4ce73466'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:46.221530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-43743bf3-7662-48b3-85ee-c92e4ce73466 admin admin] The token cache is being invalidate because project e4a81211c97b4433a63d582100871e6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:46.221830 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-43743bf3-7662-48b3-85ee-c92e4ce73466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:46.224754 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-43743bf3-7662-48b3-85ee-c92e4ce73466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:46.227447 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 874/1750] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:45 2022] DELETE /identity/v3/projects/e4a81211c97b4433a63d582100871e6a => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:46.232158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afa76b-8259-4d40-badb-fe9815df0871 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:46.232449 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afa76b-8259-4d40-badb-fe9815df0871 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:46.232728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afa76b-8259-4d40-badb-fe9815df0871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:46.257635 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d2afa76b-8259-4d40-badb-fe9815df0871 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:46.259195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d2afa76b-8259-4d40-badb-fe9815df0871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:46.340024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfbde5-e24e-453f-a63e-2ae5dc2c5ab6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:46.340431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfbde5-e24e-453f-a63e-2ae5dc2c5ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:46.340809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfbde5-e24e-453f-a63e-2ae5dc2c5ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:46.345166 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:46.380309 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 877/1751] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:46.385117 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afa76b-8259-4d40-badb-fe9815df0871 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:46.390471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d2afa76b-8259-4d40-badb-fe9815df0871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:46.398732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e1dfbde5-e24e-453f-a63e-2ae5dc2c5ab6 None None] MFA Rules not processed for user `3ee31a15cc0a481a85a8d0e161217df4`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:46.400146 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e1dfbde5-e24e-453f-a63e-2ae5dc2c5ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:46.625650 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afa76b-8259-4d40-badb-fe9815df0871 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:46.636434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 875/1752] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:25:46.648629 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d509e-e5d1-434d-8d1a-dbd77ee2fd9a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:46.657992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d509e-e5d1-434d-8d1a-dbd77ee2fd9a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:46.658717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d509e-e5d1-434d-8d1a-dbd77ee2fd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:46.662189 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949f791-42a5-4aef-b7c6-a01df87d0b7d tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a11c3c58a34a019b86e5e58740fbf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63eed118baf74e0da41a241504d33c92', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:46.662593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949f791-42a5-4aef-b7c6-a01df87d0b7d tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:46.673146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949f791-42a5-4aef-b7c6-a01df87d0b7d tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:46.674677 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949f791-42a5-4aef-b7c6-a01df87d0b7d tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:46.675468 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949f791-42a5-4aef-b7c6-a01df87d0b7d tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:46.676423 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949f791-42a5-4aef-b7c6-a01df87d0b7d tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:46.678898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949f791-42a5-4aef-b7c6-a01df87d0b7d tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:46.703333 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 878/1753] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:46.704813 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9acd8-d024-4d48-b66b-c499f716aab4 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:46.705211 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9acd8-d024-4d48-b66b-c499f716aab4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:46.705591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9acd8-d024-4d48-b66b-c499f716aab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:46.730631 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c0d9acd8-d024-4d48-b66b-c499f716aab4 None None] MFA Rules not processed for user `3ee31a15cc0a481a85a8d0e161217df4`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:46.738092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0c3d509e-e5d1-434d-8d1a-dbd77ee2fd9a None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:46.739597 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0c3d509e-e5d1-434d-8d1a-dbd77ee2fd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:46.741162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c0d9acd8-d024-4d48-b66b-c499f716aab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:46.758796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 879/1754] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:46.760536 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9acd8-d024-4d48-b66b-c499f716aab4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:46.768024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 876/1755] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:46.770474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d509e-e5d1-434d-8d1a-dbd77ee2fd9a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:46.788071 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9acd8-d024-4d48-b66b-c499f716aab4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:46.790204 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d509e-e5d1-434d-8d1a-dbd77ee2fd9a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:46.797527 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f6d08-18ea-45ee-858c-82dd04a3b326 tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a11c3c58a34a019b86e5e58740fbf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63eed118baf74e0da41a241504d33c92', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:46.798156 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f6d08-18ea-45ee-858c-82dd04a3b326 tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:46.798508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f6d08-18ea-45ee-858c-82dd04a3b326 tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:46.798786 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f6d08-18ea-45ee-858c-82dd04a3b326 tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:46.804072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaca4f2-f5d5-468b-ad25-d7ad09f1d22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:46.804936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaca4f2-f5d5-468b-ad25-d7ad09f1d22f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:46.805433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaca4f2-f5d5-468b-ad25-d7ad09f1d22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:46.805718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaca4f2-f5d5-468b-ad25-d7ad09f1d22f admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:46.806441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaca4f2-f5d5-468b-ad25-d7ad09f1d22f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:46.807322 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaca4f2-f5d5-468b-ad25-d7ad09f1d22f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:46.807950 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f6d08-18ea-45ee-858c-82dd04a3b326 tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:46.812426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaca4f2-f5d5-468b-ad25-d7ad09f1d22f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:46.813245 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f6d08-18ea-45ee-858c-82dd04a3b326 tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:46.851123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 877/1756] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:46.853108 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaca4f2-f5d5-468b-ad25-d7ad09f1d22f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:46.874884 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 880/1757] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:46 2022] POST /identity/v3/projects => generated 354 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:46.881080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f6d08-18ea-45ee-858c-82dd04a3b326 tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:46.925256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaca4f2-f5d5-468b-ad25-d7ad09f1d22f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:47.009482 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:47.040255 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f6d08-18ea-45ee-858c-82dd04a3b326 tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:47.055239 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56632039-aad9-45e8-afee-90740a997e9f tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a11c3c58a34a019b86e5e58740fbf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63eed118baf74e0da41a241504d33c92', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:47.056042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56632039-aad9-45e8-afee-90740a997e9f tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:47.056329 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56632039-aad9-45e8-afee-90740a997e9f tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:47.057235 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56632039-aad9-45e8-afee-90740a997e9f tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:47.070838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56632039-aad9-45e8-afee-90740a997e9f tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:47.071957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56632039-aad9-45e8-afee-90740a997e9f tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:47.077693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56632039-aad9-45e8-afee-90740a997e9f tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:47.090934 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4decd-86bb-4af8-9e7c-7fd3a0cd0ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:47.091566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4decd-86bb-4af8-9e7c-7fd3a0cd0ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:47.091854 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4decd-86bb-4af8-9e7c-7fd3a0cd0ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:47.094680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4decd-86bb-4af8-9e7c-7fd3a0cd0ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:47.095452 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4decd-86bb-4af8-9e7c-7fd3a0cd0ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:47.102639 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4decd-86bb-4af8-9e7c-7fd3a0cd0ee3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:47.123445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 881/1758] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:25:47.125102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56632039-aad9-45e8-afee-90740a997e9f tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:47.132474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-56632039-aad9-45e8-afee-90740a997e9f tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:47.179532 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56632039-aad9-45e8-afee-90740a997e9f tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:47.196733 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927d6b3-0e1a-47c3-b019-c536e8b8d0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:47.196961 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 878/1759] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:46 2022] POST /identity/v3/projects => generated 407 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:47.197044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927d6b3-0e1a-47c3-b019-c536e8b8d0c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:47.197044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927d6b3-0e1a-47c3-b019-c536e8b8d0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:47.197044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927d6b3-0e1a-47c3-b019-c536e8b8d0c5 admin admin] PATH_INFO: `/v3/projects/7abac19f224a481ea5cae1008061f6b8` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:47.198893 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4decd-86bb-4af8-9e7c-7fd3a0cd0ee3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:47.200032 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927d6b3-0e1a-47c3-b019-c536e8b8d0c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7abac19f224a481ea5cae1008061f6b8)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:47.206357 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927d6b3-0e1a-47c3-b019-c536e8b8d0c5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:47.342514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:47.375621 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8927d6b3-0e1a-47c3-b019-c536e8b8d0c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7abac19f224a481ea5cae1008061f6b8', 'request_id': 'req-8927d6b3-0e1a-47c3-b019-c536e8b8d0c5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:47.419133 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8927d6b3-0e1a-47c3-b019-c536e8b8d0c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7abac19f224a481ea5cae1008061f6b8', 'request_id': 'req-8927d6b3-0e1a-47c3-b019-c536e8b8d0c5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:47.458202 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8927d6b3-0e1a-47c3-b019-c536e8b8d0c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7abac19f224a481ea5cae1008061f6b8', 'request_id': 'req-8927d6b3-0e1a-47c3-b019-c536e8b8d0c5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:47.472069 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4decd-86bb-4af8-9e7c-7fd3a0cd0ee3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:47.526777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8927d6b3-0e1a-47c3-b019-c536e8b8d0c5 admin admin] The token cache is being invalidate because project 7abac19f224a481ea5cae1008061f6b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:47.532356 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8927d6b3-0e1a-47c3-b019-c536e8b8d0c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:47.535284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8927d6b3-0e1a-47c3-b019-c536e8b8d0c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:47.539336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 882/1760] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:47 2022] DELETE /identity/v3/projects/7abac19f224a481ea5cae1008061f6b8 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:47.540310 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927d6b3-0e1a-47c3-b019-c536e8b8d0c5 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:47.544524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8927d6b3-0e1a-47c3-b019-c536e8b8d0c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:47.657412 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927d6b3-0e1a-47c3-b019-c536e8b8d0c5 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:47.674680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c574d8-2dfb-4790-a420-8bfa045d9bfe tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a11c3c58a34a019b86e5e58740fbf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63eed118baf74e0da41a241504d33c92', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:47.676999 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c574d8-2dfb-4790-a420-8bfa045d9bfe tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:47.676999 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c574d8-2dfb-4790-a420-8bfa045d9bfe tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:47.677258 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c574d8-2dfb-4790-a420-8bfa045d9bfe tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] PATH_INFO: `/v3/projects/b262cb54efe745f6a9808a76e4eb5a24` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:47.679508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c574d8-2dfb-4790-a420-8bfa045d9bfe tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b262cb54efe745f6a9808a76e4eb5a24)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:47.682082 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c574d8-2dfb-4790-a420-8bfa045d9bfe tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:47.687965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4875b23-2578-4474-8ac1-0b67c5820cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:47.689458 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4875b23-2578-4474-8ac1-0b67c5820cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:47.689751 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4875b23-2578-4474-8ac1-0b67c5820cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:47.690023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4875b23-2578-4474-8ac1-0b67c5820cc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:47.690680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4875b23-2578-4474-8ac1-0b67c5820cc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:47.692632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4875b23-2578-4474-8ac1-0b67c5820cc8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:47.754850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 879/1761] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:47 2022] POST /identity/v3/users => generated 304 bytes in 557 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:47.756147 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee18379-a309-4f89-a769-7ae6fcdabea7 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:47.756488 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee18379-a309-4f89-a769-7ae6fcdabea7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:47.756777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee18379-a309-4f89-a769-7ae6fcdabea7 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:47.772467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 880/1762] 10.222.0.39 () {54 vars in 912 bytes} [Mon Aug 8 02:25:47 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:47.775219 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04fbff-6ad8-4a62-96bf-d6ff0fb6d841 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:47.775766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04fbff-6ad8-4a62-96bf-d6ff0fb6d841 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:47.778372 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04fbff-6ad8-4a62-96bf-d6ff0fb6d841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:47.815084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-16c574d8-2dfb-4790-a420-8bfa045d9bfe tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b262cb54efe745f6a9808a76e4eb5a24', 'request_id': 'req-16c574d8-2dfb-4790-a420-8bfa045d9bfe'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:47.831579 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-16c574d8-2dfb-4790-a420-8bfa045d9bfe tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b262cb54efe745f6a9808a76e4eb5a24', 'request_id': 'req-16c574d8-2dfb-4790-a420-8bfa045d9bfe'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:47.838398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3b04fbff-6ad8-4a62-96bf-d6ff0fb6d841 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:47.842153 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3b04fbff-6ad8-4a62-96bf-d6ff0fb6d841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:47.852940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-16c574d8-2dfb-4790-a420-8bfa045d9bfe tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b262cb54efe745f6a9808a76e4eb5a24', 'request_id': 'req-16c574d8-2dfb-4790-a420-8bfa045d9bfe'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:47.883794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-16c574d8-2dfb-4790-a420-8bfa045d9bfe tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] The token cache is being invalidate because project b262cb54efe745f6a9808a76e4eb5a24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:47.884145 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-16c574d8-2dfb-4790-a420-8bfa045d9bfe tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:47.885693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-16c574d8-2dfb-4790-a420-8bfa045d9bfe tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:47.891767 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 883/1763] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:47 2022] DELETE /identity/v3/projects/b262cb54efe745f6a9808a76e4eb5a24 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:47.892663 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c574d8-2dfb-4790-a420-8bfa045d9bfe tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:47.899152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-16c574d8-2dfb-4790-a420-8bfa045d9bfe tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:47.976438 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:48.084473 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 881/1764] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:48.104129 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057f5f9-92eb-4713-ab6b-57e262009018 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:48.106660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057f5f9-92eb-4713-ab6b-57e262009018 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:48.107102 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057f5f9-92eb-4713-ab6b-57e262009018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:48.179955 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0057f5f9-92eb-4713-ab6b-57e262009018 None None] MFA Rules not processed for user `78506fea1c024b169a4b7ee86116658c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:48.192042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0057f5f9-92eb-4713-ab6b-57e262009018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:48.192248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c574d8-2dfb-4790-a420-8bfa045d9bfe tempest-VolumeQuotaClassesTest-394076904 tempest-VolumeQuotaClassesTest-394076904-project-admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:48.207365 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f1952d-018b-40d5-9ff7-fc56f8004345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:48.208006 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1952d-018b-40d5-9ff7-fc56f8004345 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:48.209318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1952d-018b-40d5-9ff7-fc56f8004345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:48.209586 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1952d-018b-40d5-9ff7-fc56f8004345 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:48.211876 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f1952d-018b-40d5-9ff7-fc56f8004345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:48.215988 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f1952d-018b-40d5-9ff7-fc56f8004345 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:48.232157 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:48.250682 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 884/1765] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:48.252495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba698a2-44df-47aa-8e23-fc6c378ec5f9 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:48.255047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba698a2-44df-47aa-8e23-fc6c378ec5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:48.255491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba698a2-44df-47aa-8e23-fc6c378ec5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:48.347336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 882/1766] 10.222.0.39 () {58 vars in 1009 bytes} [Mon Aug 8 02:25:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:48.354238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add77065-1b19-44a7-82b7-dd09471b8082 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:48.354587 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add77065-1b19-44a7-82b7-dd09471b8082 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:48.354867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add77065-1b19-44a7-82b7-dd09471b8082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:48.405320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-add77065-1b19-44a7-82b7-dd09471b8082 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:48.411897 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-add77065-1b19-44a7-82b7-dd09471b8082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:48.447922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-4ba698a2-44df-47aa-8e23-fc6c378ec5f9 None None] MFA Rules not processed for user `b88ccd3e5c4c48d3a27f4eaa7874e28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:48.449231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4ba698a2-44df-47aa-8e23-fc6c378ec5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:48.467255 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 883/1767] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:48.468266 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add77065-1b19-44a7-82b7-dd09471b8082 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:48.499169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add77065-1b19-44a7-82b7-dd09471b8082 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:48.532121 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd28c8-06cb-41e7-b140-174c2e19ba43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:48.532734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd28c8-06cb-41e7-b140-174c2e19ba43 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:48.533051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd28c8-06cb-41e7-b140-174c2e19ba43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:48.533273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd28c8-06cb-41e7-b140-174c2e19ba43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:48.533933 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd28c8-06cb-41e7-b140-174c2e19ba43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:48.535596 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd28c8-06cb-41e7-b140-174c2e19ba43 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:48.573450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 884/1768] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:48.574315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd28c8-06cb-41e7-b140-174c2e19ba43 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:48.580688 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-22cd28c8-06cb-41e7-b140-174c2e19ba43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:48.620520 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd28c8-06cb-41e7-b140-174c2e19ba43 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:48.632524 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98828edf-2084-4462-8e5d-3655ff6c44ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:48.633230 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98828edf-2084-4462-8e5d-3655ff6c44ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:48.633516 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98828edf-2084-4462-8e5d-3655ff6c44ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:48.633786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98828edf-2084-4462-8e5d-3655ff6c44ae admin admin] PATH_INFO: `/v3/users/cc2e26059372409e9f6e263942063e15` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:48.639128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98828edf-2084-4462-8e5d-3655ff6c44ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc2e26059372409e9f6e263942063e15)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:48.645784 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98828edf-2084-4462-8e5d-3655ff6c44ae admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:48.649847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 885/1769] 10.222.0.39 () {58 vars in 1009 bytes} [Mon Aug 8 02:25:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:48.652893 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba698a2-44df-47aa-8e23-fc6c378ec5f9 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:48.656781 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4ba698a2-44df-47aa-8e23-fc6c378ec5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:48.688639 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba698a2-44df-47aa-8e23-fc6c378ec5f9 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:48.702480 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07828d-3f8d-4952-84a8-13fd4b96de94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:48.703017 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07828d-3f8d-4952-84a8-13fd4b96de94 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:48.703267 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07828d-3f8d-4952-84a8-13fd4b96de94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:48.703809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07828d-3f8d-4952-84a8-13fd4b96de94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:48.725598 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07828d-3f8d-4952-84a8-13fd4b96de94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:48.729544 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07828d-3f8d-4952-84a8-13fd4b96de94 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:48.755917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 886/1770] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:48.757026 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07828d-3f8d-4952-84a8-13fd4b96de94 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:48.757264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-98828edf-2084-4462-8e5d-3655ff6c44ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc2e26059372409e9f6e263942063e15'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:48.778775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-98828edf-2084-4462-8e5d-3655ff6c44ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc2e26059372409e9f6e263942063e15'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:48.795175 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-98828edf-2084-4462-8e5d-3655ff6c44ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc2e26059372409e9f6e263942063e15'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:48.796527 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-98828edf-2084-4462-8e5d-3655ff6c44ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc2e26059372409e9f6e263942063e15'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:48.797368 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-98828edf-2084-4462-8e5d-3655ff6c44ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc2e26059372409e9f6e263942063e15'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:48.805591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07828d-3f8d-4952-84a8-13fd4b96de94 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:48.823027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b78372-51b9-438a-bb9c-98cb3cb5394d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:48.825157 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b78372-51b9-438a-bb9c-98cb3cb5394d admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:48.825491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b78372-51b9-438a-bb9c-98cb3cb5394d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:48.825893 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b78372-51b9-438a-bb9c-98cb3cb5394d admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:48.830712 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b78372-51b9-438a-bb9c-98cb3cb5394d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:48.831196 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b78372-51b9-438a-bb9c-98cb3cb5394d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:48.833032 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b78372-51b9-438a-bb9c-98cb3cb5394d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:48.864723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 887/1771] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:48.868744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b78372-51b9-438a-bb9c-98cb3cb5394d admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:48.872520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-07b78372-51b9-438a-bb9c-98cb3cb5394d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:48.874435 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 885/1772] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:48 2022] DELETE /identity/v3/users/cc2e26059372409e9f6e263942063e15 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:48.875468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98828edf-2084-4462-8e5d-3655ff6c44ae admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:48.889525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-98828edf-2084-4462-8e5d-3655ff6c44ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:48.905738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b78372-51b9-438a-bb9c-98cb3cb5394d admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:49.031831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:49.073443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98828edf-2084-4462-8e5d-3655ff6c44ae admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:49.096967 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918290bc-c498-4109-bf53-98652ff85a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:49.096967 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918290bc-c498-4109-bf53-98652ff85a8b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:49.097324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918290bc-c498-4109-bf53-98652ff85a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:49.097667 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918290bc-c498-4109-bf53-98652ff85a8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:49.098406 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918290bc-c498-4109-bf53-98652ff85a8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:49.100517 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918290bc-c498-4109-bf53-98652ff85a8b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:49.101320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367229e-4532-409b-8cfb-288fc15360fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:49.102007 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367229e-4532-409b-8cfb-288fc15360fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:49.102327 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367229e-4532-409b-8cfb-288fc15360fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:49.102606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367229e-4532-409b-8cfb-288fc15360fa admin admin] PATH_INFO: `/v3/projects/946af5bb215c43869d3cf63210cc8940/users/a2d0a5cfd07c437c8194cd89282f0715/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:49.137612 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367229e-4532-409b-8cfb-288fc15360fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=946af5bb215c43869d3cf63210cc8940, user_id=a2d0a5cfd07c437c8194cd89282f0715, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:49.141013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367229e-4532-409b-8cfb-288fc15360fa admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:49.174024 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 888/1773] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:48 2022] PUT /identity/v3/projects/946af5bb215c43869d3cf63210cc8940/users/a2d0a5cfd07c437c8194cd89282f0715/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:49.175827 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03f248-a747-4998-adad-1b667064a5c7 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:49.176281 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03f248-a747-4998-adad-1b667064a5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:49.176719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03f248-a747-4998-adad-1b667064a5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:49.193655 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 886/1774] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:48 2022] POST /identity/v3/projects => generated 365 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:49.197194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918290bc-c498-4109-bf53-98652ff85a8b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:49.204177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-918290bc-c498-4109-bf53-98652ff85a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:49.234506 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-dc03f248-a747-4998-adad-1b667064a5c7 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:49.235783 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dc03f248-a747-4998-adad-1b667064a5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:49.294755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:49.390480 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 889/1775] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:49.391551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03f248-a747-4998-adad-1b667064a5c7 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:49.392986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dc03f248-a747-4998-adad-1b667064a5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:49.455057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918290bc-c498-4109-bf53-98652ff85a8b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:49.463784 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a94ef-33ff-49bd-8596-7d424f65b437 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac60b4d621234225830c94edfac400e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:49.464834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a94ef-33ff-49bd-8596-7d424f65b437 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:49.465357 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a94ef-33ff-49bd-8596-7d424f65b437 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:49.465836 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a94ef-33ff-49bd-8596-7d424f65b437 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:49.468915 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-370a94ef-33ff-49bd-8596-7d424f65b437 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:49.587060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03f248-a747-4998-adad-1b667064a5c7 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:49.595450 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c633c8f-eee0-4162-826b-0f90f9aed464 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:49.596798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c633c8f-eee0-4162-826b-0f90f9aed464 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:49.596798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c633c8f-eee0-4162-826b-0f90f9aed464 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:49.596798 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c633c8f-eee0-4162-826b-0f90f9aed464 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:49.599685 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3c633c8f-eee0-4162-826b-0f90f9aed464 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:49.652345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a94ef-33ff-49bd-8596-7d424f65b437 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:49.654064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a94ef-33ff-49bd-8596-7d424f65b437 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:49.698789 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 887/1776] 10.222.0.39 () {60 vars in 1384 bytes} [Mon Aug 8 02:25:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:25:49.715648 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddf7ad-9b0e-4478-ae68-f3527ae02419 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:49.731698 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddf7ad-9b0e-4478-ae68-f3527ae02419 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:49.731974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddf7ad-9b0e-4478-ae68-f3527ae02419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:49.786221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c633c8f-eee0-4162-826b-0f90f9aed464 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:49.794236 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c633c8f-eee0-4162-826b-0f90f9aed464 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:49.794455 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-42ddf7ad-9b0e-4478-ae68-f3527ae02419 None None] MFA Rules not processed for user `a2d0a5cfd07c437c8194cd89282f0715`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:49.795479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-42ddf7ad-9b0e-4478-ae68-f3527ae02419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:49.855710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 890/1777] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:49.856780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c633c8f-eee0-4162-826b-0f90f9aed464 service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:49.925385 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c633c8f-eee0-4162-826b-0f90f9aed464 service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:49.945728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3da21e-6068-4ac7-a156-4b25e2abb7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:49.946407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3da21e-6068-4ac7-a156-4b25e2abb7e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:49.946681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3da21e-6068-4ac7-a156-4b25e2abb7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:49.946984 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3da21e-6068-4ac7-a156-4b25e2abb7e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:49.947681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3da21e-6068-4ac7-a156-4b25e2abb7e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:49.949593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3da21e-6068-4ac7-a156-4b25e2abb7e7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:49.968132 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:50.047253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 891/1778] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:49 2022] POST /identity/v3/users => generated 283 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:50.048276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3da21e-6068-4ac7-a156-4b25e2abb7e7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:50.063183 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8c3da21e-6068-4ac7-a156-4b25e2abb7e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:50.082037 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 888/1779] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:49 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:50.083411 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc00fe-4dbd-4621-bf9b-727487a74791 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.083721 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc00fe-4dbd-4621-bf9b-727487a74791 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.083972 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc00fe-4dbd-4621-bf9b-727487a74791 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:50.085470 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 889/1780] 10.222.0.39 () {54 vars in 921 bytes} [Mon Aug 8 02:25:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:50.097903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd666d8b-ad8b-49b9-9af7-f5f3820f26b2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.098188 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd666d8b-ad8b-49b9-9af7-f5f3820f26b2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.098490 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd666d8b-ad8b-49b9-9af7-f5f3820f26b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:50.177574 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cd666d8b-ad8b-49b9-9af7-f5f3820f26b2 None None] MFA Rules not processed for user `78506fea1c024b169a4b7ee86116658c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:50.178735 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cd666d8b-ad8b-49b9-9af7-f5f3820f26b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:50.265898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:50.279622 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3da21e-6068-4ac7-a156-4b25e2abb7e7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:50.296836 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d15e62-e0e6-40bb-a50d-32f3d00c7a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:50.297460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d15e62-e0e6-40bb-a50d-32f3d00c7a3d service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.297744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d15e62-e0e6-40bb-a50d-32f3d00c7a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.298068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d15e62-e0e6-40bb-a50d-32f3d00c7a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:50.326590 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d15e62-e0e6-40bb-a50d-32f3d00c7a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:50.328029 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d15e62-e0e6-40bb-a50d-32f3d00c7a3d service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:50.350906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 892/1781] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:50.351854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d15e62-e0e6-40bb-a50d-32f3d00c7a3d service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:50.396692 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 890/1782] 10.222.0.13 () {58 vars in 1015 bytes} [Mon Aug 8 02:25:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:50.397389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d15e62-e0e6-40bb-a50d-32f3d00c7a3d service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:50.398954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd666d8b-ad8b-49b9-9af7-f5f3820f26b2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:50.418580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78542aa0-cf43-411c-a677-24229e07b839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:50.419609 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78542aa0-cf43-411c-a677-24229e07b839 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.420030 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78542aa0-cf43-411c-a677-24229e07b839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.420456 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78542aa0-cf43-411c-a677-24229e07b839 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:50.421428 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78542aa0-cf43-411c-a677-24229e07b839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:50.425045 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78542aa0-cf43-411c-a677-24229e07b839 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:50.454613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd666d8b-ad8b-49b9-9af7-f5f3820f26b2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:50.465915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 893/1783] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:50.473237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871fe66-96d4-439e-a116-9098dfe011a4 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.473237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871fe66-96d4-439e-a116-9098dfe011a4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.473237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871fe66-96d4-439e-a116-9098dfe011a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:50.504452 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b72b2-d1c4-4d0d-b07a-de278c6f8ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:50.505817 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b72b2-d1c4-4d0d-b07a-de278c6f8ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.506366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b72b2-d1c4-4d0d-b07a-de278c6f8ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.507930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b72b2-d1c4-4d0d-b07a-de278c6f8ec6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:50.507930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b72b2-d1c4-4d0d-b07a-de278c6f8ec6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:50.524435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-4871fe66-96d4-439e-a116-9098dfe011a4 None None] MFA Rules not processed for user `f62745b92a864480a9438de581e1d05e`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:50.528084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b72b2-d1c4-4d0d-b07a-de278c6f8ec6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:50.531194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4871fe66-96d4-439e-a116-9098dfe011a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:50.650861 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 891/1784] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:50 2022] POST /identity/v3/projects => generated 407 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:50.651985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b72b2-d1c4-4d0d-b07a-de278c6f8ec6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:50.677630 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 894/1785] 10.222.0.39 () {58 vars in 1018 bytes} [Mon Aug 8 02:25:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:50.678472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871fe66-96d4-439e-a116-9098dfe011a4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:50.692628 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871fe66-96d4-439e-a116-9098dfe011a4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:50.792921 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97774030-2688-430e-8240-ac5304934691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:50.793585 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97774030-2688-430e-8240-ac5304934691 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.803450 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97774030-2688-430e-8240-ac5304934691 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.803791 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97774030-2688-430e-8240-ac5304934691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:50.814632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97774030-2688-430e-8240-ac5304934691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:50.816472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97774030-2688-430e-8240-ac5304934691 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:50.828605 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 895/1786] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:50.829899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97774030-2688-430e-8240-ac5304934691 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:50.854285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b72b2-d1c4-4d0d-b07a-de278c6f8ec6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:50.859569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b50be-c33c-4aec-ac4a-be9ef8d9907c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:50.860359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b50be-c33c-4aec-ac4a-be9ef8d9907c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.860704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b50be-c33c-4aec-ac4a-be9ef8d9907c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.861053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b50be-c33c-4aec-ac4a-be9ef8d9907c admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:50.861778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b50be-c33c-4aec-ac4a-be9ef8d9907c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:50.864616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b50be-c33c-4aec-ac4a-be9ef8d9907c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:50.880918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 892/1787] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:50.882059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b50be-c33c-4aec-ac4a-be9ef8d9907c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:50.884364 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a85b50be-c33c-4aec-ac4a-be9ef8d9907c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:50.961337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97774030-2688-430e-8240-ac5304934691 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:50.973179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377ec1f-34b1-4b63-9ab8-f8228714d41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:50.974950 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377ec1f-34b1-4b63-9ab8-f8228714d41a admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.975922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377ec1f-34b1-4b63-9ab8-f8228714d41a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.976746 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377ec1f-34b1-4b63-9ab8-f8228714d41a admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:50.978542 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377ec1f-34b1-4b63-9ab8-f8228714d41a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:50.979877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b50be-c33c-4aec-ac4a-be9ef8d9907c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:50.984626 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377ec1f-34b1-4b63-9ab8-f8228714d41a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:50.995927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a511ce3-65b5-478f-8287-b931591704a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f62745b92a864480a9438de581e1d05e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:50.996597 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a511ce3-65b5-478f-8287-b931591704a4 service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.998597 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a511ce3-65b5-478f-8287-b931591704a4 service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.999589 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a511ce3-65b5-478f-8287-b931591704a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:51.013778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a511ce3-65b5-478f-8287-b931591704a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:51.017278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a511ce3-65b5-478f-8287-b931591704a4 service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:51.023266 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:51.042045 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 893/1788] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:51.043237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a511ce3-65b5-478f-8287-b931591704a4 service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:51.053251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 896/1789] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:50 2022] POST /identity/v3/users => generated 310 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:51.057882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377ec1f-34b1-4b63-9ab8-f8228714d41a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:51.090607 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a511ce3-65b5-478f-8287-b931591704a4 service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:51.098953 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377ec1f-34b1-4b63-9ab8-f8228714d41a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:51.099852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7cd95a-fd7b-46fe-975c-c292a297177a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:51.100729 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7cd95a-fd7b-46fe-975c-c292a297177a admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:51.103211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7cd95a-fd7b-46fe-975c-c292a297177a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:51.104196 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7cd95a-fd7b-46fe-975c-c292a297177a admin admin] PATH_INFO: `/v3/projects/fa8f8d4df1454f8bba189311a462f80a/users/15be06238d1647fab7e845920530e26f/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:51.118394 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597e332-92b1-4b69-a6ed-2c99c6e44749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:51.121473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597e332-92b1-4b69-a6ed-2c99c6e44749 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:51.123806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597e332-92b1-4b69-a6ed-2c99c6e44749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:51.124218 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597e332-92b1-4b69-a6ed-2c99c6e44749 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:51.129813 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597e332-92b1-4b69-a6ed-2c99c6e44749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:51.141075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597e332-92b1-4b69-a6ed-2c99c6e44749 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:51.144260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:51.155075 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7cd95a-fd7b-46fe-975c-c292a297177a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa8f8d4df1454f8bba189311a462f80a, user_id=15be06238d1647fab7e845920530e26f, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:51.157845 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7cd95a-fd7b-46fe-975c-c292a297177a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:51.163030 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 897/1790] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:51.164373 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597e332-92b1-4b69-a6ed-2c99c6e44749 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:51.176838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597e332-92b1-4b69-a6ed-2c99c6e44749 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:51.191221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 894/1791] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:51 2022] PUT /identity/v3/projects/fa8f8d4df1454f8bba189311a462f80a/users/15be06238d1647fab7e845920530e26f/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:51.193998 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85af4a-b174-4beb-936a-7466cef2cb98 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:51.194472 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85af4a-b174-4beb-936a-7466cef2cb98 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:51.195020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85af4a-b174-4beb-936a-7466cef2cb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:51.248053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ce85af4a-b174-4beb-936a-7466cef2cb98 None None] MFA Rules not processed for user `15be06238d1647fab7e845920530e26f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:51.269350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ce85af4a-b174-4beb-936a-7466cef2cb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:51.384881 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fce13-0937-47f8-9f21-bd8f783a4e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:51.388170 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fce13-0937-47f8-9f21-bd8f783a4e81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:51.389977 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fce13-0937-47f8-9f21-bd8f783a4e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:51.391011 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fce13-0937-47f8-9f21-bd8f783a4e81 admin admin] PATH_INFO: `/v3/projects/43dc70d1ed8b4dc5a682309f4c44a0c0/users/4462e8c191554902b494d275c38cf97a/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:51.448911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9fce13-0937-47f8-9f21-bd8f783a4e81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43dc70d1ed8b4dc5a682309f4c44a0c0, user_id=4462e8c191554902b494d275c38cf97a, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:51.455882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9fce13-0937-47f8-9f21-bd8f783a4e81 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:51.507340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 895/1792] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:51.508492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85af4a-b174-4beb-936a-7466cef2cb98 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:51.508991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 898/1793] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:51 2022] PUT /identity/v3/projects/43dc70d1ed8b4dc5a682309f4c44a0c0/users/4462e8c191554902b494d275c38cf97a/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:51.513956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fce13-0937-47f8-9f21-bd8f783a4e81 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:51.532626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85af4a-b174-4beb-936a-7466cef2cb98 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:51.649146 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021a571-0931-45d6-8a86-fb8da33c7b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:51.649610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021a571-0931-45d6-8a86-fb8da33c7b97 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:51.650894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021a571-0931-45d6-8a86-fb8da33c7b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:51.651243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021a571-0931-45d6-8a86-fb8da33c7b97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:51.651961 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8021a571-0931-45d6-8a86-fb8da33c7b97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:51.656001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8021a571-0931-45d6-8a86-fb8da33c7b97 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:51.711541 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 896/1794] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:51 2022] POST /identity/v3/projects => generated 365 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:51.718981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021a571-0931-45d6-8a86-fb8da33c7b97 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:51.772940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fce13-0937-47f8-9f21-bd8f783a4e81 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:51.885605 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:51.953922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf955de-083f-4fb8-adda-f48cf1995f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:51.954589 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf955de-083f-4fb8-adda-f48cf1995f04 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:51.954885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf955de-083f-4fb8-adda-f48cf1995f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:51.955142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf955de-083f-4fb8-adda-f48cf1995f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:51.955813 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf955de-083f-4fb8-adda-f48cf1995f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:51.957460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf955de-083f-4fb8-adda-f48cf1995f04 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:51.965349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021a571-0931-45d6-8a86-fb8da33c7b97 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:51.981483 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a39a7-bfd2-41b4-8529-0f37592315b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:51.982095 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a39a7-bfd2-41b4-8529-0f37592315b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:51.982443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a39a7-bfd2-41b4-8529-0f37592315b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:51.984601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a39a7-bfd2-41b4-8529-0f37592315b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:51.985286 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a39a7-bfd2-41b4-8529-0f37592315b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:51.987384 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a39a7-bfd2-41b4-8529-0f37592315b0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:51.990306 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 899/1795] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 477 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:52.001821 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf955de-083f-4fb8-adda-f48cf1995f04 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:52.040745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf955de-083f-4fb8-adda-f48cf1995f04 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:52.045071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 897/1796] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:51 2022] POST /identity/v3/users => generated 289 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:52.047382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a39a7-bfd2-41b4-8529-0f37592315b0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:52.061979 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545cfd8a-82a6-4611-a4df-76b710970dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:52.062697 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545cfd8a-82a6-4611-a4df-76b710970dc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:52.062979 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545cfd8a-82a6-4611-a4df-76b710970dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:52.063257 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545cfd8a-82a6-4611-a4df-76b710970dc5 admin admin] PATH_INFO: `/v3/projects/43dc70d1ed8b4dc5a682309f4c44a0c0/users/4462e8c191554902b494d275c38cf97a/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:52.072341 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545cfd8a-82a6-4611-a4df-76b710970dc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43dc70d1ed8b4dc5a682309f4c44a0c0, user_id=4462e8c191554902b494d275c38cf97a, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:52.075319 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545cfd8a-82a6-4611-a4df-76b710970dc5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:52.085609 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a39a7-bfd2-41b4-8529-0f37592315b0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:52.097029 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 900/1797] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:51 2022] PUT /identity/v3/projects/43dc70d1ed8b4dc5a682309f4c44a0c0/users/4462e8c191554902b494d275c38cf97a/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:52.100020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b162465-dd0c-4a24-85c3-8bb368332d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:52.100651 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b162465-dd0c-4a24-85c3-8bb368332d77 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:52.100933 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b162465-dd0c-4a24-85c3-8bb368332d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:52.101203 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b162465-dd0c-4a24-85c3-8bb368332d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:52.101854 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b162465-dd0c-4a24-85c3-8bb368332d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:52.104025 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b162465-dd0c-4a24-85c3-8bb368332d77 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:52.114715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f765d-e75e-4e33-bbb9-a0d9f0e49bb9 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:52.115145 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f765d-e75e-4e33-bbb9-a0d9f0e49bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:52.115746 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f765d-e75e-4e33-bbb9-a0d9f0e49bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:52.134586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 898/1798] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:52.143725 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b162465-dd0c-4a24-85c3-8bb368332d77 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:52.168472 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b162465-dd0c-4a24-85c3-8bb368332d77 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:52.191195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:52.232466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-314f765d-e75e-4e33-bbb9-a0d9f0e49bb9 None None] MFA Rules not processed for user `4462e8c191554902b494d275c38cf97a`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:52.240654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-314f765d-e75e-4e33-bbb9-a0d9f0e49bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:52.312261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718848f-cb5c-4c83-a706-e978a7f4839e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:52.313341 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718848f-cb5c-4c83-a706-e978a7f4839e admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:52.314220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718848f-cb5c-4c83-a706-e978a7f4839e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:52.315140 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718848f-cb5c-4c83-a706-e978a7f4839e admin admin] PATH_INFO: `/v3/projects/f17bb745e2734a55965696115e3785fa/users/e09ef8aa66b544c880c9debfa4be49aa/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:52.345285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718848f-cb5c-4c83-a706-e978a7f4839e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f17bb745e2734a55965696115e3785fa, user_id=e09ef8aa66b544c880c9debfa4be49aa, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:52.348659 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718848f-cb5c-4c83-a706-e978a7f4839e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:52.393773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 899/1799] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:52 2022] PUT /identity/v3/projects/f17bb745e2734a55965696115e3785fa/users/e09ef8aa66b544c880c9debfa4be49aa/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:52.401758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718848f-cb5c-4c83-a706-e978a7f4839e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:52.513991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 901/1800] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:52 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:52.549186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f765d-e75e-4e33-bbb9-a0d9f0e49bb9 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:52.551662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-314f765d-e75e-4e33-bbb9-a0d9f0e49bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:52.647689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718848f-cb5c-4c83-a706-e978a7f4839e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:52.662623 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06444b16-efb5-455f-a0b9-7fdff45f2b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:52.663252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06444b16-efb5-455f-a0b9-7fdff45f2b69 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:52.663865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06444b16-efb5-455f-a0b9-7fdff45f2b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:52.664138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06444b16-efb5-455f-a0b9-7fdff45f2b69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:52.668207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06444b16-efb5-455f-a0b9-7fdff45f2b69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:52.671530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06444b16-efb5-455f-a0b9-7fdff45f2b69 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:52.772104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 900/1801] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:52.774612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06444b16-efb5-455f-a0b9-7fdff45f2b69 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:52.810488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f765d-e75e-4e33-bbb9-a0d9f0e49bb9 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:52.812461 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06444b16-efb5-455f-a0b9-7fdff45f2b69 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:52.840964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ec30a-e049-4489-8022-f3c4b0fbd66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:52.843454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ec30a-e049-4489-8022-f3c4b0fbd66d admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:52.843746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ec30a-e049-4489-8022-f3c4b0fbd66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:52.844026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ec30a-e049-4489-8022-f3c4b0fbd66d admin admin] PATH_INFO: `/v3/projects/f17bb745e2734a55965696115e3785fa/users/e09ef8aa66b544c880c9debfa4be49aa/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:52.855325 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ec30a-e049-4489-8022-f3c4b0fbd66d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f17bb745e2734a55965696115e3785fa, user_id=e09ef8aa66b544c880c9debfa4be49aa, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:52.858416 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ec30a-e049-4489-8022-f3c4b0fbd66d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:52.928782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 901/1802] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:52 2022] PUT /identity/v3/projects/f17bb745e2734a55965696115e3785fa/users/e09ef8aa66b544c880c9debfa4be49aa/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:52.963698 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0206d-e9e0-400b-b37b-6d9d6d4c047c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:52.963976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0206d-e9e0-400b-b37b-6d9d6d4c047c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:52.964248 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0206d-e9e0-400b-b37b-6d9d6d4c047c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:53.029505 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf3af6-03a9-4743-a7a5-79537ba33145 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:53.031400 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf3af6-03a9-4743-a7a5-79537ba33145 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:53.032383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf3af6-03a9-4743-a7a5-79537ba33145 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:53.032667 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf3af6-03a9-4743-a7a5-79537ba33145 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:53.107257 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8aa0206d-e9e0-400b-b37b-6d9d6d4c047c None None] MFA Rules not processed for user `e09ef8aa66b544c880c9debfa4be49aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:53.108625 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8aa0206d-e9e0-400b-b37b-6d9d6d4c047c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:53.219831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:53.281515 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf3af6-03a9-4743-a7a5-79537ba33145 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:53.289223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf3af6-03a9-4743-a7a5-79537ba33145 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:53.354221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 902/1803] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:52 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 812 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:53.461152 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 902/1804] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 492 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:53.462277 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf3af6-03a9-4743-a7a5-79537ba33145 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:53.465904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ccbf3af6-03a9-4743-a7a5-79537ba33145 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:53.478931 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:53.493312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf3af6-03a9-4743-a7a5-79537ba33145 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:53.639005 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3a910-6c68-4ec8-8b0e-ac52684bc3a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:53.640499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3a910-6c68-4ec8-8b0e-ac52684bc3a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:53.641799 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3a910-6c68-4ec8-8b0e-ac52684bc3a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:53.642099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3a910-6c68-4ec8-8b0e-ac52684bc3a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:53.670507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3a910-6c68-4ec8-8b0e-ac52684bc3a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:53.676901 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3a910-6c68-4ec8-8b0e-ac52684bc3a7 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:53.716478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 903/1805] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:53 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:25:53.860730 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0206d-e9e0-400b-b37b-6d9d6d4c047c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:53.875070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0206d-e9e0-400b-b37b-6d9d6d4c047c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:53.907346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68f733-37fd-44bc-8e32-f02137a59899 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:53.924089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68f733-37fd-44bc-8e32-f02137a59899 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:53.937189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68f733-37fd-44bc-8e32-f02137a59899 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:53.937721 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68f733-37fd-44bc-8e32-f02137a59899 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:54.278437 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68f733-37fd-44bc-8e32-f02137a59899 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:54.280047 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68f733-37fd-44bc-8e32-f02137a59899 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:54.313837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 903/1806] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:53 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:54.756050 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad1af5-920a-46fa-b7de-6a5bd591b9bb None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:54.758937 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad1af5-920a-46fa-b7de-6a5bd591b9bb None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:54.760559 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad1af5-920a-46fa-b7de-6a5bd591b9bb None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:54.763104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 904/1807] 10.222.0.39 () {54 vars in 921 bytes} [Mon Aug 8 02:25:54 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:54.850567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874c284-da0a-4530-a011-0af591c41068 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:54.851261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874c284-da0a-4530-a011-0af591c41068 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:54.851762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874c284-da0a-4530-a011-0af591c41068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:54.927104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6874c284-da0a-4530-a011-0af591c41068 None None] MFA Rules not processed for user `f62745b92a864480a9438de581e1d05e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:54.929874 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6874c284-da0a-4530-a011-0af591c41068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:54.955963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad1af5-920a-46fa-b7de-6a5bd591b9bb None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:54.959644 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9bad1af5-920a-46fa-b7de-6a5bd591b9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:55.141991 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 904/1808] 10.222.0.39 () {58 vars in 1018 bytes} [Mon Aug 8 02:25:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:55.232235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874c284-da0a-4530-a011-0af591c41068 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:55.234078 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6874c284-da0a-4530-a011-0af591c41068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:55.256395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874c284-da0a-4530-a011-0af591c41068 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:55.276398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ba375-6c89-4238-acb8-7e55194a06a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f62745b92a864480a9438de581e1d05e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:55.277291 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ba375-6c89-4238-acb8-7e55194a06a1 service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:55.277730 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ba375-6c89-4238-acb8-7e55194a06a1 service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:55.278196 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ba375-6c89-4238-acb8-7e55194a06a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:55.297532 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ba375-6c89-4238-acb8-7e55194a06a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:55.302707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ba375-6c89-4238-acb8-7e55194a06a1 service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:55.353030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 905/1809] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:25:55.367158 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ba375-6c89-4238-acb8-7e55194a06a1 service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:55.397557 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ba375-6c89-4238-acb8-7e55194a06a1 service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:55.399227 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad1af5-920a-46fa-b7de-6a5bd591b9bb None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:55.423414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bde950-2c91-439f-af91-a5d5b5f4954a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:55.425813 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57880f0e-7c52-4769-9bea-e56c1136a667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:55.427644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bde950-2c91-439f-af91-a5d5b5f4954a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:55.428170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bde950-2c91-439f-af91-a5d5b5f4954a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:55.429025 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57880f0e-7c52-4769-9bea-e56c1136a667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:55.430116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57880f0e-7c52-4769-9bea-e56c1136a667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:55.432112 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bde950-2c91-439f-af91-a5d5b5f4954a admin admin] PATH_INFO: `/v3/users/15be06238d1647fab7e845920530e26f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:55.433725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57880f0e-7c52-4769-9bea-e56c1136a667 admin admin] PATH_INFO: `/v3/users/46ecdd9d0eb84701a651a1c687ad4522` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:55.439650 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57880f0e-7c52-4769-9bea-e56c1136a667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46ecdd9d0eb84701a651a1c687ad4522)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:55.448601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bde950-2c91-439f-af91-a5d5b5f4954a admin admin] RBAC: Authorizing `identity:delete_user(user_id=15be06238d1647fab7e845920530e26f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:55.452748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bde950-2c91-439f-af91-a5d5b5f4954a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:55.455862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57880f0e-7c52-4769-9bea-e56c1136a667 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:55.628860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:55.645366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-57880f0e-7c52-4769-9bea-e56c1136a667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46ecdd9d0eb84701a651a1c687ad4522'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:55.673873 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c6bde950-2c91-439f-af91-a5d5b5f4954a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15be06238d1647fab7e845920530e26f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:55.676265 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-57880f0e-7c52-4769-9bea-e56c1136a667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46ecdd9d0eb84701a651a1c687ad4522'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:55.692297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-57880f0e-7c52-4769-9bea-e56c1136a667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46ecdd9d0eb84701a651a1c687ad4522'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:55.693613 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-57880f0e-7c52-4769-9bea-e56c1136a667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46ecdd9d0eb84701a651a1c687ad4522'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:55.695254 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-57880f0e-7c52-4769-9bea-e56c1136a667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46ecdd9d0eb84701a651a1c687ad4522'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:55.698380 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c6bde950-2c91-439f-af91-a5d5b5f4954a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15be06238d1647fab7e845920530e26f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:55.716160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c6bde950-2c91-439f-af91-a5d5b5f4954a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15be06238d1647fab7e845920530e26f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:55.717299 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c6bde950-2c91-439f-af91-a5d5b5f4954a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15be06238d1647fab7e845920530e26f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:55.718925 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c6bde950-2c91-439f-af91-a5d5b5f4954a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15be06238d1647fab7e845920530e26f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:55.758904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 905/1810] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:54 2022] DELETE /identity/v3/users/46ecdd9d0eb84701a651a1c687ad4522 => generated 0 bytes in 800 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:55.788935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 906/1811] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:55 2022] DELETE /identity/v3/users/15be06238d1647fab7e845920530e26f => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:55.794800 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e8445-e5dd-4d58-9688-63993cb512ba None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:55.795305 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e8445-e5dd-4d58-9688-63993cb512ba None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:55.795740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e8445-e5dd-4d58-9688-63993cb512ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:55.842537 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6d0e8445-e5dd-4d58-9688-63993cb512ba None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:55.844145 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6d0e8445-e5dd-4d58-9688-63993cb512ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:55.901891 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:56.021360 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 906/1812] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:56.094477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bde950-2c91-439f-af91-a5d5b5f4954a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:56.096133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c6bde950-2c91-439f-af91-a5d5b5f4954a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:56.266757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bde950-2c91-439f-af91-a5d5b5f4954a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:56.278924 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09dc20-1154-40bc-ab25-9e4b47fdebb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:56.281567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09dc20-1154-40bc-ab25-9e4b47fdebb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:56.283002 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09dc20-1154-40bc-ab25-9e4b47fdebb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:56.284291 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09dc20-1154-40bc-ab25-9e4b47fdebb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:56.312534 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e09dc20-1154-40bc-ab25-9e4b47fdebb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:56.324942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e09dc20-1154-40bc-ab25-9e4b47fdebb9 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:56.358320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 907/1813] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:57.521622 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb66a0-9e87-4448-b1f6-b4a8ef3d68f5 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:57.522718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb66a0-9e87-4448-b1f6-b4a8ef3d68f5 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:57.523319 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb66a0-9e87-4448-b1f6-b4a8ef3d68f5 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:57.525911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 907/1814] 10.222.0.39 () {54 vars in 921 bytes} [Mon Aug 8 02:25:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:57.868228 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb77cc-4914-4997-8b0e-d2e8d192515e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:57.868751 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb77cc-4914-4997-8b0e-d2e8d192515e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:57.869090 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb77cc-4914-4997-8b0e-d2e8d192515e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:57.929413 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-91eb77cc-4914-4997-8b0e-d2e8d192515e None None] MFA Rules not processed for user `c4238566a9c248ffbfa314f47f02670c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:57.930780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-91eb77cc-4914-4997-8b0e-d2e8d192515e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:58.048571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb66a0-9e87-4448-b1f6-b4a8ef3d68f5 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:58.051591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d2fb66a0-9e87-4448-b1f6-b4a8ef3d68f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:58.102620 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb66a0-9e87-4448-b1f6-b4a8ef3d68f5 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:58.105287 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 908/1815] 10.222.0.39 () {58 vars in 1018 bytes} [Mon Aug 8 02:25:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:58.106195 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb77cc-4914-4997-8b0e-d2e8d192515e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:58.109654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-91eb77cc-4914-4997-8b0e-d2e8d192515e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:58.124233 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a9a15-a74e-4d50-8863-7eeaa45b194b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:58.124883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a9a15-a74e-4d50-8863-7eeaa45b194b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:58.125161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a9a15-a74e-4d50-8863-7eeaa45b194b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:58.125432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a9a15-a74e-4d50-8863-7eeaa45b194b admin admin] PATH_INFO: `/v3/projects/b0723e1594f94cb0a0e694b863e920fd` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:58.126911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a9a15-a74e-4d50-8863-7eeaa45b194b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0723e1594f94cb0a0e694b863e920fd)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:58.147230 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb77cc-4914-4997-8b0e-d2e8d192515e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:58.147362 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a9a15-a74e-4d50-8863-7eeaa45b194b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:58.165965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb16b2-a439-4c38-b8a8-f4adf93b113c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:58.166849 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb16b2-a439-4c38-b8a8-f4adf93b113c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:58.167142 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb16b2-a439-4c38-b8a8-f4adf93b113c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:58.167420 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb16b2-a439-4c38-b8a8-f4adf93b113c admin admin] PATH_INFO: `/v3/projects/a6ef7e48e4194a3992deb964c68af44c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:58.169492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb16b2-a439-4c38-b8a8-f4adf93b113c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6ef7e48e4194a3992deb964c68af44c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:58.171638 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb16b2-a439-4c38-b8a8-f4adf93b113c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:58.281471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-446a9a15-a74e-4d50-8863-7eeaa45b194b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0723e1594f94cb0a0e694b863e920fd', 'request_id': 'req-446a9a15-a74e-4d50-8863-7eeaa45b194b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.293017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:58.302334 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-446a9a15-a74e-4d50-8863-7eeaa45b194b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0723e1594f94cb0a0e694b863e920fd', 'request_id': 'req-446a9a15-a74e-4d50-8863-7eeaa45b194b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.304957 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-17cb16b2-a439-4c38-b8a8-f4adf93b113c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6ef7e48e4194a3992deb964c68af44c', 'request_id': 'req-17cb16b2-a439-4c38-b8a8-f4adf93b113c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.325934 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-17cb16b2-a439-4c38-b8a8-f4adf93b113c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6ef7e48e4194a3992deb964c68af44c', 'request_id': 'req-17cb16b2-a439-4c38-b8a8-f4adf93b113c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.339938 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-17cb16b2-a439-4c38-b8a8-f4adf93b113c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6ef7e48e4194a3992deb964c68af44c', 'request_id': 'req-17cb16b2-a439-4c38-b8a8-f4adf93b113c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.342938 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-446a9a15-a74e-4d50-8863-7eeaa45b194b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0723e1594f94cb0a0e694b863e920fd', 'request_id': 'req-446a9a15-a74e-4d50-8863-7eeaa45b194b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.360038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-17cb16b2-a439-4c38-b8a8-f4adf93b113c admin admin] The token cache is being invalidate because project a6ef7e48e4194a3992deb964c68af44c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:58.362611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-17cb16b2-a439-4c38-b8a8-f4adf93b113c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.363802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-17cb16b2-a439-4c38-b8a8-f4adf93b113c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.367817 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 909/1816] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:58 2022] DELETE /identity/v3/projects/a6ef7e48e4194a3992deb964c68af44c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:58.380516 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb16b2-a439-4c38-b8a8-f4adf93b113c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:58.382020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-17cb16b2-a439-4c38-b8a8-f4adf93b113c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:58.398925 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-446a9a15-a74e-4d50-8863-7eeaa45b194b admin admin] The token cache is being invalidate because project b0723e1594f94cb0a0e694b863e920fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:58.399244 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-446a9a15-a74e-4d50-8863-7eeaa45b194b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.401300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-446a9a15-a74e-4d50-8863-7eeaa45b194b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.405003 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 908/1817] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:58 2022] DELETE /identity/v3/projects/b0723e1594f94cb0a0e694b863e920fd => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:58.405924 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a9a15-a74e-4d50-8863-7eeaa45b194b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:58.407538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-446a9a15-a74e-4d50-8863-7eeaa45b194b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:58.516065 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb16b2-a439-4c38-b8a8-f4adf93b113c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:58.531566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ca99d-ca31-4c4d-8457-a0f21a1bbcb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:58.532240 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ca99d-ca31-4c4d-8457-a0f21a1bbcb3 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:58.532523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ca99d-ca31-4c4d-8457-a0f21a1bbcb3 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:58.532794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ca99d-ca31-4c4d-8457-a0f21a1bbcb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:58.535390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a56ca99d-ca31-4c4d-8457-a0f21a1bbcb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:58.574502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ca99d-ca31-4c4d-8457-a0f21a1bbcb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:58.576151 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ca99d-ca31-4c4d-8457-a0f21a1bbcb3 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:58.604087 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a9a15-a74e-4d50-8863-7eeaa45b194b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:58.606652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 910/1818] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:58.608395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ca99d-ca31-4c4d-8457-a0f21a1bbcb3 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:58.611128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a56ca99d-ca31-4c4d-8457-a0f21a1bbcb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:58.729981 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:58.758148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac20b93-1129-4810-898c-c2c6a19dbdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:58.760900 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac20b93-1129-4810-898c-c2c6a19dbdfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:58.761475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac20b93-1129-4810-898c-c2c6a19dbdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:58.762137 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac20b93-1129-4810-898c-c2c6a19dbdfa admin admin] PATH_INFO: `/v3/users/e1a11c3c58a34a019b86e5e58740fbf9` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:58.764516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac20b93-1129-4810-898c-c2c6a19dbdfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1a11c3c58a34a019b86e5e58740fbf9)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:58.766992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac20b93-1129-4810-898c-c2c6a19dbdfa admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:58.792743 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ca99d-ca31-4c4d-8457-a0f21a1bbcb3 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:58.803616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b28b7-e546-49f1-a635-e48f03178df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:58.804632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b28b7-e546-49f1-a635-e48f03178df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:58.805114 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b28b7-e546-49f1-a635-e48f03178df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:58.805495 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b28b7-e546-49f1-a635-e48f03178df0 admin admin] PATH_INFO: `/v3/users/a4bd5e92711b4737be207f3b235a7992` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:58.807528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b28b7-e546-49f1-a635-e48f03178df0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4bd5e92711b4737be207f3b235a7992)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:58.810383 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b28b7-e546-49f1-a635-e48f03178df0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:58.892501 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-034b28b7-e546-49f1-a635-e48f03178df0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4bd5e92711b4737be207f3b235a7992'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.905986 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-034b28b7-e546-49f1-a635-e48f03178df0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4bd5e92711b4737be207f3b235a7992'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.912582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cac20b93-1129-4810-898c-c2c6a19dbdfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1a11c3c58a34a019b86e5e58740fbf9'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.913424 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:58.921326 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cac20b93-1129-4810-898c-c2c6a19dbdfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1a11c3c58a34a019b86e5e58740fbf9'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.926813 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-034b28b7-e546-49f1-a635-e48f03178df0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4bd5e92711b4737be207f3b235a7992'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.928546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-034b28b7-e546-49f1-a635-e48f03178df0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4bd5e92711b4737be207f3b235a7992'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.929306 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-034b28b7-e546-49f1-a635-e48f03178df0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4bd5e92711b4737be207f3b235a7992'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.930944 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cac20b93-1129-4810-898c-c2c6a19dbdfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1a11c3c58a34a019b86e5e58740fbf9'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.932396 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cac20b93-1129-4810-898c-c2c6a19dbdfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1a11c3c58a34a019b86e5e58740fbf9'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.933543 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cac20b93-1129-4810-898c-c2c6a19dbdfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1a11c3c58a34a019b86e5e58740fbf9'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.955514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 911/1819] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:58 2022] DELETE /identity/v3/users/a4bd5e92711b4737be207f3b235a7992 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:58.955514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b28b7-e546-49f1-a635-e48f03178df0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:58.961480 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 909/1820] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:58 2022] DELETE /identity/v3/users/e1a11c3c58a34a019b86e5e58740fbf9 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:58.962455 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-034b28b7-e546-49f1-a635-e48f03178df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:59.045531 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b28b7-e546-49f1-a635-e48f03178df0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:59.156415 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44554d32-da39-4e70-bdbe-df4cf6c3f0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:59.157582 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44554d32-da39-4e70-bdbe-df4cf6c3f0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:59.158023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44554d32-da39-4e70-bdbe-df4cf6c3f0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:59.159163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44554d32-da39-4e70-bdbe-df4cf6c3f0cc admin admin] PATH_INFO: `/v3/projects/fa8f8d4df1454f8bba189311a462f80a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:59.161732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44554d32-da39-4e70-bdbe-df4cf6c3f0cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa8f8d4df1454f8bba189311a462f80a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:59.164306 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44554d32-da39-4e70-bdbe-df4cf6c3f0cc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:59.265555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-44554d32-da39-4e70-bdbe-df4cf6c3f0cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa8f8d4df1454f8bba189311a462f80a', 'request_id': 'req-44554d32-da39-4e70-bdbe-df4cf6c3f0cc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.272239 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-44554d32-da39-4e70-bdbe-df4cf6c3f0cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa8f8d4df1454f8bba189311a462f80a', 'request_id': 'req-44554d32-da39-4e70-bdbe-df4cf6c3f0cc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.284450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-44554d32-da39-4e70-bdbe-df4cf6c3f0cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa8f8d4df1454f8bba189311a462f80a', 'request_id': 'req-44554d32-da39-4e70-bdbe-df4cf6c3f0cc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.304036 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-44554d32-da39-4e70-bdbe-df4cf6c3f0cc admin admin] The token cache is being invalidate because project fa8f8d4df1454f8bba189311a462f80a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:59.304467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-44554d32-da39-4e70-bdbe-df4cf6c3f0cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.305291 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-44554d32-da39-4e70-bdbe-df4cf6c3f0cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.309922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 912/1821] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:58 2022] DELETE /identity/v3/projects/fa8f8d4df1454f8bba189311a462f80a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:59.311826 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac20b93-1129-4810-898c-c2c6a19dbdfa admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:59.313454 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cac20b93-1129-4810-898c-c2c6a19dbdfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:59.344744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac20b93-1129-4810-898c-c2c6a19dbdfa admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:59.444426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d875b41-28c8-46f9-ace8-175635bb69d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:59.445251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d875b41-28c8-46f9-ace8-175635bb69d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:59.445541 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d875b41-28c8-46f9-ace8-175635bb69d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:59.445816 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d875b41-28c8-46f9-ace8-175635bb69d2 admin admin] PATH_INFO: `/v3/users/e09ef8aa66b544c880c9debfa4be49aa` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:59.447220 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d875b41-28c8-46f9-ace8-175635bb69d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e09ef8aa66b544c880c9debfa4be49aa)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:59.449710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d875b41-28c8-46f9-ace8-175635bb69d2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:59.545036 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2d875b41-28c8-46f9-ace8-175635bb69d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e09ef8aa66b544c880c9debfa4be49aa'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.552785 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2d875b41-28c8-46f9-ace8-175635bb69d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e09ef8aa66b544c880c9debfa4be49aa'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.567248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2d875b41-28c8-46f9-ace8-175635bb69d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e09ef8aa66b544c880c9debfa4be49aa'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.568639 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2d875b41-28c8-46f9-ace8-175635bb69d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e09ef8aa66b544c880c9debfa4be49aa'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.569533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2d875b41-28c8-46f9-ace8-175635bb69d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e09ef8aa66b544c880c9debfa4be49aa'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.584726 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:59.600590 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 910/1822] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:59 2022] DELETE /identity/v3/users/e09ef8aa66b544c880c9debfa4be49aa => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:59.780092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44554d32-da39-4e70-bdbe-df4cf6c3f0cc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:59.784760 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-44554d32-da39-4e70-bdbe-df4cf6c3f0cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:59.805010 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d875b41-28c8-46f9-ace8-175635bb69d2 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:59.806872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2d875b41-28c8-46f9-ace8-175635bb69d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:59.815977 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44554d32-da39-4e70-bdbe-df4cf6c3f0cc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:59.850743 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:59.919700 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d875b41-28c8-46f9-ace8-175635bb69d2 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:59.924892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1782ccb-3402-42e6-96e6-7a5c3333baf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:59.925646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1782ccb-3402-42e6-96e6-7a5c3333baf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:59.925986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1782ccb-3402-42e6-96e6-7a5c3333baf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:59.926382 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1782ccb-3402-42e6-96e6-7a5c3333baf8 admin admin] PATH_INFO: `/v3/projects/63eed118baf74e0da41a241504d33c92` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:59.927642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1782ccb-3402-42e6-96e6-7a5c3333baf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63eed118baf74e0da41a241504d33c92)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:59.930030 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1782ccb-3402-42e6-96e6-7a5c3333baf8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:59.971302 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00290f-3b8d-41bb-b650-b3224a7f33e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:59.972560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00290f-3b8d-41bb-b650-b3224a7f33e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:59.972821 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00290f-3b8d-41bb-b650-b3224a7f33e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:59.973097 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00290f-3b8d-41bb-b650-b3224a7f33e6 admin admin] PATH_INFO: `/v3/projects/04b5b49139c246a1a8e0c8107c8bf4ca` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:59.974948 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00290f-3b8d-41bb-b650-b3224a7f33e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04b5b49139c246a1a8e0c8107c8bf4ca)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:59.976891 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00290f-3b8d-41bb-b650-b3224a7f33e6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:59.999257 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c1782ccb-3402-42e6-96e6-7a5c3333baf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63eed118baf74e0da41a241504d33c92', 'request_id': 'req-c1782ccb-3402-42e6-96e6-7a5c3333baf8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:00.006009 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c1782ccb-3402-42e6-96e6-7a5c3333baf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63eed118baf74e0da41a241504d33c92', 'request_id': 'req-c1782ccb-3402-42e6-96e6-7a5c3333baf8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:00.028197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c1782ccb-3402-42e6-96e6-7a5c3333baf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63eed118baf74e0da41a241504d33c92', 'request_id': 'req-c1782ccb-3402-42e6-96e6-7a5c3333baf8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:00.050049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c1782ccb-3402-42e6-96e6-7a5c3333baf8 admin admin] The token cache is being invalidate because project 63eed118baf74e0da41a241504d33c92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:00.050375 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c1782ccb-3402-42e6-96e6-7a5c3333baf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:00.051126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c1782ccb-3402-42e6-96e6-7a5c3333baf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:00.053089 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 911/1823] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:59 2022] DELETE /identity/v3/projects/63eed118baf74e0da41a241504d33c92 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:00.054492 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f6678-1b9c-476b-a7a2-eef391219fe9 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.054776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f6678-1b9c-476b-a7a2-eef391219fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.056359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f6678-1b9c-476b-a7a2-eef391219fe9 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.058208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 912/1824] 10.222.0.39 () {54 vars in 921 bytes} [Mon Aug 8 02:26:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:00.059321 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464fab5-97e6-4050-a528-e9d0da4f6313 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.059706 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464fab5-97e6-4050-a528-e9d0da4f6313 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.060125 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464fab5-97e6-4050-a528-e9d0da4f6313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.074083 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4c00290f-3b8d-41bb-b650-b3224a7f33e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04b5b49139c246a1a8e0c8107c8bf4ca', 'request_id': 'req-4c00290f-3b8d-41bb-b650-b3224a7f33e6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:00.081977 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f464fab5-97e6-4050-a528-e9d0da4f6313 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:00.089171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4c00290f-3b8d-41bb-b650-b3224a7f33e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04b5b49139c246a1a8e0c8107c8bf4ca', 'request_id': 'req-4c00290f-3b8d-41bb-b650-b3224a7f33e6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:00.090540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f464fab5-97e6-4050-a528-e9d0da4f6313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:00.112864 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4c00290f-3b8d-41bb-b650-b3224a7f33e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04b5b49139c246a1a8e0c8107c8bf4ca', 'request_id': 'req-4c00290f-3b8d-41bb-b650-b3224a7f33e6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:00.126790 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4c00290f-3b8d-41bb-b650-b3224a7f33e6 admin admin] The token cache is being invalidate because project 04b5b49139c246a1a8e0c8107c8bf4ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:00.127644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4c00290f-3b8d-41bb-b650-b3224a7f33e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:00.128572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4c00290f-3b8d-41bb-b650-b3224a7f33e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:00.130309 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 913/1825] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:59 2022] DELETE /identity/v3/projects/04b5b49139c246a1a8e0c8107c8bf4ca => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:00.135374 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b28e90-8f3d-40af-8a7f-7c4c44d13ca2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.135670 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b28e90-8f3d-40af-8a7f-7c4c44d13ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.135928 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b28e90-8f3d-40af-8a7f-7c4c44d13ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.163947 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-05b28e90-8f3d-40af-8a7f-7c4c44d13ca2 None None] MFA Rules not processed for user `c4238566a9c248ffbfa314f47f02670c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:00.165157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-05b28e90-8f3d-40af-8a7f-7c4c44d13ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:00.185981 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 913/1826] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:00.187788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222a7a1-731d-45a0-bcf9-42fb2f86790c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.188166 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222a7a1-731d-45a0-bcf9-42fb2f86790c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.188450 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222a7a1-731d-45a0-bcf9-42fb2f86790c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.208883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-0222a7a1-731d-45a0-bcf9-42fb2f86790c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:00.210344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0222a7a1-731d-45a0-bcf9-42fb2f86790c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:00.218473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 914/1827] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:00.219644 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222a7a1-731d-45a0-bcf9-42fb2f86790c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.221126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0222a7a1-731d-45a0-bcf9-42fb2f86790c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:00.234100 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222a7a1-731d-45a0-bcf9-42fb2f86790c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.239052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee7840-6971-4f1a-973e-40f935a1be1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.239679 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee7840-6971-4f1a-973e-40f935a1be1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.239960 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee7840-6971-4f1a-973e-40f935a1be1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.240873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee7840-6971-4f1a-973e-40f935a1be1f admin admin] PATH_INFO: `/v3/projects/f17bb745e2734a55965696115e3785fa` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.241623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee7840-6971-4f1a-973e-40f935a1be1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f17bb745e2734a55965696115e3785fa)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.243574 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee7840-6971-4f1a-973e-40f935a1be1f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.255638 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 914/1828] 10.222.0.39 () {58 vars in 1018 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:00.257060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7c71c-7db8-4ab8-9220-0c405014821e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.257349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7c71c-7db8-4ab8-9220-0c405014821e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.257615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7c71c-7db8-4ab8-9220-0c405014821e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.265317 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:00.281204 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e4d7c71c-7db8-4ab8-9220-0c405014821e None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:00.282736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e4d7c71c-7db8-4ab8-9220-0c405014821e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:00.293102 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 915/1829] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:00.294391 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf004f-fdfb-4e84-a63c-2a97ab65786f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.294678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf004f-fdfb-4e84-a63c-2a97ab65786f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.294954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf004f-fdfb-4e84-a63c-2a97ab65786f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.304962 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-dcee7840-6971-4f1a-973e-40f935a1be1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f17bb745e2734a55965696115e3785fa', 'request_id': 'req-dcee7840-6971-4f1a-973e-40f935a1be1f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:00.312258 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-dcee7840-6971-4f1a-973e-40f935a1be1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f17bb745e2734a55965696115e3785fa', 'request_id': 'req-dcee7840-6971-4f1a-973e-40f935a1be1f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:00.321298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-58cf004f-fdfb-4e84-a63c-2a97ab65786f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:00.322561 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-58cf004f-fdfb-4e84-a63c-2a97ab65786f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:00.323003 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-dcee7840-6971-4f1a-973e-40f935a1be1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f17bb745e2734a55965696115e3785fa', 'request_id': 'req-dcee7840-6971-4f1a-973e-40f935a1be1f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:00.338326 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-dcee7840-6971-4f1a-973e-40f935a1be1f admin admin] The token cache is being invalidate because project f17bb745e2734a55965696115e3785fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:00.338642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-dcee7840-6971-4f1a-973e-40f935a1be1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:00.339276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-dcee7840-6971-4f1a-973e-40f935a1be1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:00.341173 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 915/1830] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:00 2022] DELETE /identity/v3/projects/f17bb745e2734a55965696115e3785fa => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:00.343454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:00.345179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee7840-6971-4f1a-973e-40f935a1be1f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.346773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dcee7840-6971-4f1a-973e-40f935a1be1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:00.388993 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 916/1831] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:00.389628 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf004f-fdfb-4e84-a63c-2a97ab65786f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.390963 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-58cf004f-fdfb-4e84-a63c-2a97ab65786f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:00.404780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf004f-fdfb-4e84-a63c-2a97ab65786f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.409483 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699762d-c351-4a5b-98d2-2215ad1b16c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.410083 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699762d-c351-4a5b-98d2-2215ad1b16c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.410393 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699762d-c351-4a5b-98d2-2215ad1b16c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.410662 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699762d-c351-4a5b-98d2-2215ad1b16c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.411311 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699762d-c351-4a5b-98d2-2215ad1b16c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:00.411614 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699762d-c351-4a5b-98d2-2215ad1b16c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.413255 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699762d-c351-4a5b-98d2-2215ad1b16c1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.429367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 917/1832] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:00.430765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547fe53-62fb-4b90-97a2-58e92425035e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.431046 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547fe53-62fb-4b90-97a2-58e92425035e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.431355 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547fe53-62fb-4b90-97a2-58e92425035e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.435079 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee7840-6971-4f1a-973e-40f935a1be1f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.450346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fe4d0-cadb-4cd4-9bec-37921ee3d383 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.450346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fe4d0-cadb-4cd4-9bec-37921ee3d383 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.450346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fe4d0-cadb-4cd4-9bec-37921ee3d383 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.450592 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fe4d0-cadb-4cd4-9bec-37921ee3d383 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.451408 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5547fe53-62fb-4b90-97a2-58e92425035e None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:00.452547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2f4fe4d0-cadb-4cd4-9bec-37921ee3d383 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:00.452666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5547fe53-62fb-4b90-97a2-58e92425035e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:00.462298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 918/1833] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:00.462298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547fe53-62fb-4b90-97a2-58e92425035e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.468408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fe4d0-cadb-4cd4-9bec-37921ee3d383 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.470141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fe4d0-cadb-4cd4-9bec-37921ee3d383 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.479412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547fe53-62fb-4b90-97a2-58e92425035e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.482851 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 916/1834] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:00.483591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fe4d0-cadb-4cd4-9bec-37921ee3d383 service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.484700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684f93d-8d0f-4d81-910b-787b722a53c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.485412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684f93d-8d0f-4d81-910b-787b722a53c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.485807 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684f93d-8d0f-4d81-910b-787b722a53c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.486176 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684f93d-8d0f-4d81-910b-787b722a53c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.486948 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684f93d-8d0f-4d81-910b-787b722a53c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:00.487351 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684f93d-8d0f-4d81-910b-787b722a53c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.489183 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684f93d-8d0f-4d81-910b-787b722a53c7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.494723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fe4d0-cadb-4cd4-9bec-37921ee3d383 service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.499577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419731b-f695-4238-812d-0cc4dc158f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.500309 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419731b-f695-4238-812d-0cc4dc158f15 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.500659 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419731b-f695-4238-812d-0cc4dc158f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.501030 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419731b-f695-4238-812d-0cc4dc158f15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.501872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2419731b-f695-4238-812d-0cc4dc158f15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.505703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 919/1835] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:00.505703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561db44-4598-422d-b561-438f6f790fa0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.505703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561db44-4598-422d-b561-438f6f790fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.506201 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2419731b-f695-4238-812d-0cc4dc158f15 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.506302 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561db44-4598-422d-b561-438f6f790fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.528147 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3561db44-4598-422d-b561-438f6f790fa0 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:00.530130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3561db44-4598-422d-b561-438f6f790fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:00.540185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 920/1836] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:00.541658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561db44-4598-422d-b561-438f6f790fa0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.547196 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 917/1837] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:00.548176 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419731b-f695-4238-812d-0cc4dc158f15 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.562755 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561db44-4598-422d-b561-438f6f790fa0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.630108 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a7b6c-e070-4977-bb19-09f49398c870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.630763 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a7b6c-e070-4977-bb19-09f49398c870 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.631039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a7b6c-e070-4977-bb19-09f49398c870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.631396 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a7b6c-e070-4977-bb19-09f49398c870 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.631926 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a7b6c-e070-4977-bb19-09f49398c870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.633848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a7b6c-e070-4977-bb19-09f49398c870 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.633966 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419731b-f695-4238-812d-0cc4dc158f15 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.639940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c57dd6-f593-4ef7-a2b7-2dde87b63a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.640627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c57dd6-f593-4ef7-a2b7-2dde87b63a1b admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.640977 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c57dd6-f593-4ef7-a2b7-2dde87b63a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.641321 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c57dd6-f593-4ef7-a2b7-2dde87b63a1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.642044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c57dd6-f593-4ef7-a2b7-2dde87b63a1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:00.642447 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c57dd6-f593-4ef7-a2b7-2dde87b63a1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.644103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c57dd6-f593-4ef7-a2b7-2dde87b63a1b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.651585 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:00.658299 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 918/1838] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:00.659128 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c57dd6-f593-4ef7-a2b7-2dde87b63a1b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.672509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 921/1839] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:00.673467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c57dd6-f593-4ef7-a2b7-2dde87b63a1b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.673715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a7b6c-e070-4977-bb19-09f49398c870 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.737879 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cb368-7894-4060-b529-b72c8aadc351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.738525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cb368-7894-4060-b529-b72c8aadc351 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.738806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cb368-7894-4060-b529-b72c8aadc351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.739122 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cb368-7894-4060-b529-b72c8aadc351 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.739750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cb368-7894-4060-b529-b72c8aadc351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.741650 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cb368-7894-4060-b529-b72c8aadc351 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.745404 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a7b6c-e070-4977-bb19-09f49398c870 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.750414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57acc68-ff34-4221-9f04-8e04be080f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.751088 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57acc68-ff34-4221-9f04-8e04be080f5e admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.751529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57acc68-ff34-4221-9f04-8e04be080f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.751801 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57acc68-ff34-4221-9f04-8e04be080f5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.752416 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57acc68-ff34-4221-9f04-8e04be080f5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.754460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57acc68-ff34-4221-9f04-8e04be080f5e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.766941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 919/1840] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:00.768081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cb368-7894-4060-b529-b72c8aadc351 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.779223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cb368-7894-4060-b529-b72c8aadc351 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.783712 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:00.784518 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15f3b1-57ea-43c9-88bc-1588ba7f7086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.784873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15f3b1-57ea-43c9-88bc-1588ba7f7086 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.785140 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15f3b1-57ea-43c9-88bc-1588ba7f7086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.785428 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15f3b1-57ea-43c9-88bc-1588ba7f7086 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.786145 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15f3b1-57ea-43c9-88bc-1588ba7f7086 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.788141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15f3b1-57ea-43c9-88bc-1588ba7f7086 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.796764 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 922/1841] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/projects => generated 369 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:00.796764 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57acc68-ff34-4221-9f04-8e04be080f5e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.813252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 920/1842] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/users => generated 287 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:00.815127 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15f3b1-57ea-43c9-88bc-1588ba7f7086 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.826529 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15f3b1-57ea-43c9-88bc-1588ba7f7086 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.867234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57acc68-ff34-4221-9f04-8e04be080f5e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.872567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7729e-5fea-4517-be1e-d01dd205d852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.873179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7729e-5fea-4517-be1e-d01dd205d852 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.873457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7729e-5fea-4517-be1e-d01dd205d852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.873730 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7729e-5fea-4517-be1e-d01dd205d852 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.874421 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7729e-5fea-4517-be1e-d01dd205d852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.876085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7729e-5fea-4517-be1e-d01dd205d852 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.889780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 923/1843] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:00.889780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7729e-5fea-4517-be1e-d01dd205d852 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.895314 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d2f02-b184-4d73-bf7c-fda2fe1bea4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.895932 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d2f02-b184-4d73-bf7c-fda2fe1bea4a admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.896208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d2f02-b184-4d73-bf7c-fda2fe1bea4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.896485 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d2f02-b184-4d73-bf7c-fda2fe1bea4a admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.897114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d2f02-b184-4d73-bf7c-fda2fe1bea4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.898737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7729e-5fea-4517-be1e-d01dd205d852 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.899053 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d2f02-b184-4d73-bf7c-fda2fe1bea4a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.904368 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7aaea-b623-4eba-bbd7-ed2d059f63b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.904965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7aaea-b623-4eba-bbd7-ed2d059f63b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.905245 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7aaea-b623-4eba-bbd7-ed2d059f63b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.905518 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7aaea-b623-4eba-bbd7-ed2d059f63b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.906158 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7aaea-b623-4eba-bbd7-ed2d059f63b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.907839 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7aaea-b623-4eba-bbd7-ed2d059f63b2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.920405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 924/1844] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:00.921235 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 921/1845] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:00.921364 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7aaea-b623-4eba-bbd7-ed2d059f63b2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.925393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d2f02-b184-4d73-bf7c-fda2fe1bea4a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.935692 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d2f02-b184-4d73-bf7c-fda2fe1bea4a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.937618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7aaea-b623-4eba-bbd7-ed2d059f63b2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.941002 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cd3f7-315b-42f7-9b7e-400b5d5d2fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.941748 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cd3f7-315b-42f7-9b7e-400b5d5d2fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.942037 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cd3f7-315b-42f7-9b7e-400b5d5d2fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.942360 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cd3f7-315b-42f7-9b7e-400b5d5d2fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.943013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5cd3f7-315b-42f7-9b7e-400b5d5d2fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.943135 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da3ff8-0f28-4f78-85a5-d4ebf639b8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.943725 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da3ff8-0f28-4f78-85a5-d4ebf639b8ac admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.944011 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da3ff8-0f28-4f78-85a5-d4ebf639b8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.944284 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da3ff8-0f28-4f78-85a5-d4ebf639b8ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.944768 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5cd3f7-315b-42f7-9b7e-400b5d5d2fd4 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.944917 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da3ff8-0f28-4f78-85a5-d4ebf639b8ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.946759 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da3ff8-0f28-4f78-85a5-d4ebf639b8ac admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.956854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 922/1846] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:00.959176 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cd3f7-315b-42f7-9b7e-400b5d5d2fd4 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.960676 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 925/1847] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:00.961520 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da3ff8-0f28-4f78-85a5-d4ebf639b8ac admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.970898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cd3f7-315b-42f7-9b7e-400b5d5d2fd4 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.972006 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da3ff8-0f28-4f78-85a5-d4ebf639b8ac admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.975473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4a069-f4f8-48e1-899c-b9256e298768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.976053 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4a069-f4f8-48e1-899c-b9256e298768 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.976330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4a069-f4f8-48e1-899c-b9256e298768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.976598 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4a069-f4f8-48e1-899c-b9256e298768 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.976687 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6293d4e-b1b2-4d77-946e-c3a7ca786cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.977229 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4a069-f4f8-48e1-899c-b9256e298768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.977327 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6293d4e-b1b2-4d77-946e-c3a7ca786cae admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.977536 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6293d4e-b1b2-4d77-946e-c3a7ca786cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.977810 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6293d4e-b1b2-4d77-946e-c3a7ca786cae admin admin] PATH_INFO: `/v3/projects/e96f7ae0385a42c79010a3bfae291da5/users/e6840cd650924821a2681c4b0f80b63e/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.978861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4a069-f4f8-48e1-899c-b9256e298768 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.989023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 923/1848] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:00.989802 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4a069-f4f8-48e1-899c-b9256e298768 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.993168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6293d4e-b1b2-4d77-946e-c3a7ca786cae admin admin] RBAC: Authorizing `identity:create_grant(project_id=e96f7ae0385a42c79010a3bfae291da5, user_id=e6840cd650924821a2681c4b0f80b63e, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.995887 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6293d4e-b1b2-4d77-946e-c3a7ca786cae admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.001102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4a069-f4f8-48e1-899c-b9256e298768 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.006213 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3270af-02be-46a2-a97b-7d55769c2d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.006903 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3270af-02be-46a2-a97b-7d55769c2d00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.007178 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3270af-02be-46a2-a97b-7d55769c2d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.007445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3270af-02be-46a2-a97b-7d55769c2d00 admin admin] PATH_INFO: `/v3/projects/a7a0ae80f10046d1bd9fd4a03e8251dc/users/1d569f36ac59420788cb2bebf03218a3/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.008770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 926/1849] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:00 2022] PUT /identity/v3/projects/e96f7ae0385a42c79010a3bfae291da5/users/e6840cd650924821a2681c4b0f80b63e/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:01.010126 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6293d4e-b1b2-4d77-946e-c3a7ca786cae admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.022963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3270af-02be-46a2-a97b-7d55769c2d00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a0ae80f10046d1bd9fd4a03e8251dc, user_id=1d569f36ac59420788cb2bebf03218a3, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.026330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3270af-02be-46a2-a97b-7d55769c2d00 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.039138 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 924/1850] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:00 2022] PUT /identity/v3/projects/a7a0ae80f10046d1bd9fd4a03e8251dc/users/1d569f36ac59420788cb2bebf03218a3/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:01.040378 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af9bdf-0855-4dab-86cd-3a5044743b23 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.040713 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af9bdf-0855-4dab-86cd-3a5044743b23 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.041123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af9bdf-0855-4dab-86cd-3a5044743b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.067265 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:01.075873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d4af9bdf-0855-4dab-86cd-3a5044743b23 None None] MFA Rules not processed for user `e6840cd650924821a2681c4b0f80b63e`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:01.077136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d4af9bdf-0855-4dab-86cd-3a5044743b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:01.084195 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6293d4e-b1b2-4d77-946e-c3a7ca786cae admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.140057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:01.149630 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb80b3-c1a5-4cee-b426-de6aa2c08152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.150771 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb80b3-c1a5-4cee-b426-de6aa2c08152 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.151114 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb80b3-c1a5-4cee-b426-de6aa2c08152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.151714 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb80b3-c1a5-4cee-b426-de6aa2c08152 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.152777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb80b3-c1a5-4cee-b426-de6aa2c08152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.154981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb80b3-c1a5-4cee-b426-de6aa2c08152 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.163488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 925/1851] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:01.163488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62788608-d2db-4139-b23a-9efd1fe579a5 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.163488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62788608-d2db-4139-b23a-9efd1fe579a5 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.163488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62788608-d2db-4139-b23a-9efd1fe579a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.167296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 927/1852] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:01.168171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb80b3-c1a5-4cee-b426-de6aa2c08152 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.179259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb80b3-c1a5-4cee-b426-de6aa2c08152 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.186226 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b380b-9baf-4551-8ae9-46e45a2778fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.186485 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-62788608-d2db-4139-b23a-9efd1fe579a5 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:01.187128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b380b-9baf-4551-8ae9-46e45a2778fd admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.187584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b380b-9baf-4551-8ae9-46e45a2778fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.187919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b380b-9baf-4551-8ae9-46e45a2778fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.188467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-62788608-d2db-4139-b23a-9efd1fe579a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:01.188618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b380b-9baf-4551-8ae9-46e45a2778fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.190935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b380b-9baf-4551-8ae9-46e45a2778fd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.200713 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 926/1853] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:01.200713 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62788608-d2db-4139-b23a-9efd1fe579a5 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.210608 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62788608-d2db-4139-b23a-9efd1fe579a5 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.216842 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56545fc4-08d9-42b8-a94b-c815e5f0fe42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.218176 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56545fc4-08d9-42b8-a94b-c815e5f0fe42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.218490 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56545fc4-08d9-42b8-a94b-c815e5f0fe42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.218771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56545fc4-08d9-42b8-a94b-c815e5f0fe42 admin admin] PATH_INFO: `/v3/projects/df806e6b8bc14a5dab778572144ca71b/users/622831a733034b1a91ae3787eab450e1/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.227428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 928/1854] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:01 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:01.229194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b380b-9baf-4551-8ae9-46e45a2778fd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.231250 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-489b380b-9baf-4551-8ae9-46e45a2778fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:01.235956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56545fc4-08d9-42b8-a94b-c815e5f0fe42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df806e6b8bc14a5dab778572144ca71b, user_id=622831a733034b1a91ae3787eab450e1, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.239472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56545fc4-08d9-42b8-a94b-c815e5f0fe42 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.257097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 927/1855] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:01 2022] PUT /identity/v3/projects/df806e6b8bc14a5dab778572144ca71b/users/622831a733034b1a91ae3787eab450e1/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:01.258209 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56545fc4-08d9-42b8-a94b-c815e5f0fe42 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.298978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b380b-9baf-4551-8ae9-46e45a2778fd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.326278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56545fc4-08d9-42b8-a94b-c815e5f0fe42 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.331509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0d0fd-2b3b-4096-9c66-2d70e7a6c075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.332854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0d0fd-2b3b-4096-9c66-2d70e7a6c075 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.333137 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0d0fd-2b3b-4096-9c66-2d70e7a6c075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.333578 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0d0fd-2b3b-4096-9c66-2d70e7a6c075 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.334239 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0d0fd-2b3b-4096-9c66-2d70e7a6c075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.336636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0d0fd-2b3b-4096-9c66-2d70e7a6c075 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.355326 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b857c-0372-45c0-abce-6637877a76e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.355943 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b857c-0372-45c0-abce-6637877a76e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.356225 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b857c-0372-45c0-abce-6637877a76e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.356501 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b857c-0372-45c0-abce-6637877a76e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.359345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 928/1856] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:01 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:01.360790 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa67dc2-6e95-41fb-bb8f-e56abf7e0195 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.361074 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa67dc2-6e95-41fb-bb8f-e56abf7e0195 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.361375 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa67dc2-6e95-41fb-bb8f-e56abf7e0195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.366927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b857c-0372-45c0-abce-6637877a76e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.368367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b857c-0372-45c0-abce-6637877a76e1 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.378536 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 929/1857] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:01.384217 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b857c-0372-45c0-abce-6637877a76e1 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.389894 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6aa67dc2-6e95-41fb-bb8f-e56abf7e0195 None None] MFA Rules not processed for user `622831a733034b1a91ae3787eab450e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:01.392382 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6aa67dc2-6e95-41fb-bb8f-e56abf7e0195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:01.396360 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b857c-0372-45c0-abce-6637877a76e1 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.402600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45409298-0a55-4013-b7c5-9ee0f52776ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.403204 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45409298-0a55-4013-b7c5-9ee0f52776ac admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.403478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45409298-0a55-4013-b7c5-9ee0f52776ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.403742 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45409298-0a55-4013-b7c5-9ee0f52776ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.404373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45409298-0a55-4013-b7c5-9ee0f52776ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.405972 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45409298-0a55-4013-b7c5-9ee0f52776ac admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.420245 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 930/1858] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:01.421119 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45409298-0a55-4013-b7c5-9ee0f52776ac admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.431067 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45409298-0a55-4013-b7c5-9ee0f52776ac admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.436222 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadac850-adc5-43eb-ad61-32ad0d6d6aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.436897 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadac850-adc5-43eb-ad61-32ad0d6d6aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.437141 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadac850-adc5-43eb-ad61-32ad0d6d6aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.437411 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadac850-adc5-43eb-ad61-32ad0d6d6aa3 admin admin] PATH_INFO: `/v3/projects/bd33b18093a349d594d63c892df79314/users/5cfa8214286f49a8b41ccf341f73be62/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.461865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadac850-adc5-43eb-ad61-32ad0d6d6aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd33b18093a349d594d63c892df79314, user_id=5cfa8214286f49a8b41ccf341f73be62, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.465527 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadac850-adc5-43eb-ad61-32ad0d6d6aa3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.483001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:01.484494 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 931/1859] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:01 2022] PUT /identity/v3/projects/bd33b18093a349d594d63c892df79314/users/5cfa8214286f49a8b41ccf341f73be62/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:01.485569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadac850-adc5-43eb-ad61-32ad0d6d6aa3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.506521 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 929/1860] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:01.512772 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa67dc2-6e95-41fb-bb8f-e56abf7e0195 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.524072 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa67dc2-6e95-41fb-bb8f-e56abf7e0195 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.551278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:01.565847 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadac850-adc5-43eb-ad61-32ad0d6d6aa3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.571475 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f9c38-3bda-43ba-8ee6-bb3ecd300e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.572616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f9c38-3bda-43ba-8ee6-bb3ecd300e72 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.572894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f9c38-3bda-43ba-8ee6-bb3ecd300e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.573165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f9c38-3bda-43ba-8ee6-bb3ecd300e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.573816 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f9c38-3bda-43ba-8ee6-bb3ecd300e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.575479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f9c38-3bda-43ba-8ee6-bb3ecd300e72 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.587071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 932/1861] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:01.592645 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f9c38-3bda-43ba-8ee6-bb3ecd300e72 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.604991 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f9c38-3bda-43ba-8ee6-bb3ecd300e72 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.607462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855feb74-d89d-4aac-a313-976cfed62fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.608103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855feb74-d89d-4aac-a313-976cfed62fbd admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.608378 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855feb74-d89d-4aac-a313-976cfed62fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.608649 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855feb74-d89d-4aac-a313-976cfed62fbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.609275 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855feb74-d89d-4aac-a313-976cfed62fbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.609976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f8f84-b27f-4f49-94d2-bd32ae34d1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.610668 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f8f84-b27f-4f49-94d2-bd32ae34d1dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.610945 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f8f84-b27f-4f49-94d2-bd32ae34d1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.611216 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f8f84-b27f-4f49-94d2-bd32ae34d1dc admin admin] PATH_INFO: `/v3/projects/bd33b18093a349d594d63c892df79314/users/5cfa8214286f49a8b41ccf341f73be62/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.611396 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855feb74-d89d-4aac-a313-976cfed62fbd admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.613102 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f8f84-b27f-4f49-94d2-bd32ae34d1dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd33b18093a349d594d63c892df79314, user_id=5cfa8214286f49a8b41ccf341f73be62, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.615748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f8f84-b27f-4f49-94d2-bd32ae34d1dc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.628344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 933/1862] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:01 2022] PUT /identity/v3/projects/bd33b18093a349d594d63c892df79314/users/5cfa8214286f49a8b41ccf341f73be62/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:01.639248 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcc8ee-726f-460a-86d8-3c733cb67310 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.639548 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcc8ee-726f-460a-86d8-3c733cb67310 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.639879 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcc8ee-726f-460a-86d8-3c733cb67310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.647866 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 930/1863] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:01 2022] POST /identity/v3/projects => generated 369 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:01.649127 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855feb74-d89d-4aac-a313-976cfed62fbd admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.672130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9cdcc8ee-726f-460a-86d8-3c733cb67310 None None] MFA Rules not processed for user `5cfa8214286f49a8b41ccf341f73be62`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:01.673348 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9cdcc8ee-726f-460a-86d8-3c733cb67310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:01.753508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855feb74-d89d-4aac-a313-976cfed62fbd admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.758830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1ee37-b6c5-4790-bf59-3154b3c1420a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.758991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1ee37-b6c5-4790-bf59-3154b3c1420a admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.759263 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1ee37-b6c5-4790-bf59-3154b3c1420a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.759576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1ee37-b6c5-4790-bf59-3154b3c1420a admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.760565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1ee37-b6c5-4790-bf59-3154b3c1420a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.762612 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1ee37-b6c5-4790-bf59-3154b3c1420a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.790521 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 931/1864] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:01 2022] POST /identity/v3/users => generated 291 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:01.797820 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1ee37-b6c5-4790-bf59-3154b3c1420a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.798387 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 934/1865] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:01.805699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdcc8ee-726f-460a-86d8-3c733cb67310 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.808060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9cdcc8ee-726f-460a-86d8-3c733cb67310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:01.816618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1ee37-b6c5-4790-bf59-3154b3c1420a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.821725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41d43b-ed3a-4322-9665-3f6352b45044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.822511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41d43b-ed3a-4322-9665-3f6352b45044 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.822905 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41d43b-ed3a-4322-9665-3f6352b45044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.823320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41d43b-ed3a-4322-9665-3f6352b45044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.828759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41d43b-ed3a-4322-9665-3f6352b45044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.830581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41d43b-ed3a-4322-9665-3f6352b45044 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.843699 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 932/1866] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:01.847874 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41d43b-ed3a-4322-9665-3f6352b45044 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.866190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41d43b-ed3a-4322-9665-3f6352b45044 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.880540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec48d23-a727-4849-8c07-1f59ad7be538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.881326 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec48d23-a727-4849-8c07-1f59ad7be538 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.881616 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec48d23-a727-4849-8c07-1f59ad7be538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.881929 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec48d23-a727-4849-8c07-1f59ad7be538 admin admin] PATH_INFO: `/v3/projects/b90e999421f54753a58707e43d672bde/users/e7caeead95e143e9b8cfe257101e6d00/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.900454 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec48d23-a727-4849-8c07-1f59ad7be538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b90e999421f54753a58707e43d672bde, user_id=e7caeead95e143e9b8cfe257101e6d00, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.903327 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec48d23-a727-4849-8c07-1f59ad7be538 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.939258 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 933/1867] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:01 2022] PUT /identity/v3/projects/b90e999421f54753a58707e43d672bde/users/e7caeead95e143e9b8cfe257101e6d00/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:01.940468 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec48d23-a727-4849-8c07-1f59ad7be538 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.943239 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdcc8ee-726f-460a-86d8-3c733cb67310 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.984639 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:02.017025 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31a945-ec6e-4ddc-bd36-ed86b266ebae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:02.017747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31a945-ec6e-4ddc-bd36-ed86b266ebae service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:02.018124 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31a945-ec6e-4ddc-bd36-ed86b266ebae service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:02.018607 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31a945-ec6e-4ddc-bd36-ed86b266ebae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:02.025932 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec48d23-a727-4849-8c07-1f59ad7be538 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:02.031706 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a1def-f6c1-4972-bfe6-9d0825d6a4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:02.032310 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a1def-f6c1-4972-bfe6-9d0825d6a4f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:02.032588 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a1def-f6c1-4972-bfe6-9d0825d6a4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:02.032856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a1def-f6c1-4972-bfe6-9d0825d6a4f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:02.033495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a1def-f6c1-4972-bfe6-9d0825d6a4f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:02.035618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a1def-f6c1-4972-bfe6-9d0825d6a4f5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:02.047809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 934/1868] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:02.051883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a1def-f6c1-4972-bfe6-9d0825d6a4f5 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:02.053898 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:02.063699 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a1def-f6c1-4972-bfe6-9d0825d6a4f5 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:02.069965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfeafc4-6571-4fd6-89a0-735d7393a286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:02.070715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeafc4-6571-4fd6-89a0-735d7393a286 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:02.070926 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeafc4-6571-4fd6-89a0-735d7393a286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:02.071200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeafc4-6571-4fd6-89a0-735d7393a286 admin admin] PATH_INFO: `/v3/projects/b90e999421f54753a58707e43d672bde/users/e7caeead95e143e9b8cfe257101e6d00/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:02.074156 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfeafc4-6571-4fd6-89a0-735d7393a286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b90e999421f54753a58707e43d672bde, user_id=e7caeead95e143e9b8cfe257101e6d00, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:02.078940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfeafc4-6571-4fd6-89a0-735d7393a286 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:02.098793 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 935/1869] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:02 2022] PUT /identity/v3/projects/b90e999421f54753a58707e43d672bde/users/e7caeead95e143e9b8cfe257101e6d00/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:02.104494 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5572b2-f647-4a38-90a4-3bd12c614475 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:02.105059 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5572b2-f647-4a38-90a4-3bd12c614475 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:02.105605 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5572b2-f647-4a38-90a4-3bd12c614475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:02.115327 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31a945-ec6e-4ddc-bd36-ed86b266ebae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:02.116897 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31a945-ec6e-4ddc-bd36-ed86b266ebae service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:02.127618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 935/1870] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:02.140700 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-5a5572b2-f647-4a38-90a4-3bd12c614475 None None] MFA Rules not processed for user `e7caeead95e143e9b8cfe257101e6d00`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:02.142055 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5a5572b2-f647-4a38-90a4-3bd12c614475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:02.226165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 936/1871] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:02.244540 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31a945-ec6e-4ddc-bd36-ed86b266ebae service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:02.245534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5572b2-f647-4a38-90a4-3bd12c614475 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:02.246000 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7c31a945-ec6e-4ddc-bd36-ed86b266ebae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:02.256855 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31a945-ec6e-4ddc-bd36-ed86b266ebae service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:02.311789 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f305e0-6f58-4a3c-a75b-dee0e7377a0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:02.312401 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f305e0-6f58-4a3c-a75b-dee0e7377a0e service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:02.312672 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f305e0-6f58-4a3c-a75b-dee0e7377a0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:02.312935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f305e0-6f58-4a3c-a75b-dee0e7377a0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:02.313686 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5572b2-f647-4a38-90a4-3bd12c614475 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:02.317814 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067f972-b78a-4a8a-8be2-9c779aae774c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:02.318533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067f972-b78a-4a8a-8be2-9c779aae774c service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:02.318898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067f972-b78a-4a8a-8be2-9c779aae774c service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:02.319342 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067f972-b78a-4a8a-8be2-9c779aae774c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:02.338837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067f972-b78a-4a8a-8be2-9c779aae774c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:02.341444 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067f972-b78a-4a8a-8be2-9c779aae774c service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:02.372387 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 937/1872] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:26:02.391391 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f305e0-6f58-4a3c-a75b-dee0e7377a0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:02.393722 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f305e0-6f58-4a3c-a75b-dee0e7377a0e service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:02.411318 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 936/1873] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:02.429366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067f972-b78a-4a8a-8be2-9c779aae774c service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:02.442627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067f972-b78a-4a8a-8be2-9c779aae774c service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:02.448232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e6288-fb98-4a5a-b6e8-b5084c25de63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:02.448841 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e6288-fb98-4a5a-b6e8-b5084c25de63 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:02.449693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e6288-fb98-4a5a-b6e8-b5084c25de63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:02.449693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e6288-fb98-4a5a-b6e8-b5084c25de63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:02.503802 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:02.527952 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e6288-fb98-4a5a-b6e8-b5084c25de63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:02.529914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e6288-fb98-4a5a-b6e8-b5084c25de63 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:02.542831 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 938/1874] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:02.576796 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8744cb8-1137-405b-9e49-1d97cdba9227 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:02.577180 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8744cb8-1137-405b-9e49-1d97cdba9227 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:02.577795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8744cb8-1137-405b-9e49-1d97cdba9227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:02.634885 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b8744cb8-1137-405b-9e49-1d97cdba9227 None None] MFA Rules not processed for user `1d569f36ac59420788cb2bebf03218a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:02.636201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b8744cb8-1137-405b-9e49-1d97cdba9227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:02.744414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 937/1875] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:02.748318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e6288-fb98-4a5a-b6e8-b5084c25de63 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:02.851377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e6288-fb98-4a5a-b6e8-b5084c25de63 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:02.864045 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf596c4c-ef46-4372-ac54-82631800e2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:02.866469 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf596c4c-ef46-4372-ac54-82631800e2fa admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:02.867925 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf596c4c-ef46-4372-ac54-82631800e2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:02.868950 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf596c4c-ef46-4372-ac54-82631800e2fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:02.870683 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf596c4c-ef46-4372-ac54-82631800e2fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:02.874937 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf596c4c-ef46-4372-ac54-82631800e2fa admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:02.951230 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 939/1876] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:02 2022] POST /identity/v3/projects => generated 373 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:02.955750 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8744cb8-1137-405b-9e49-1d97cdba9227 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:02.973559 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8744cb8-1137-405b-9e49-1d97cdba9227 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:02.995210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:03.109789 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b20af-fbc9-4946-8b83-c4695db3463f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:03.110449 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b20af-fbc9-4946-8b83-c4695db3463f admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:03.110711 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b20af-fbc9-4946-8b83-c4695db3463f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:03.110983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b20af-fbc9-4946-8b83-c4695db3463f admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:03.111786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b20af-fbc9-4946-8b83-c4695db3463f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:03.130301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b20af-fbc9-4946-8b83-c4695db3463f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:03.190360 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 938/1877] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:02 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:03.193994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf596c4c-ef46-4372-ac54-82631800e2fa admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:03.207959 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf596c4c-ef46-4372-ac54-82631800e2fa admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:03.214113 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23ab76-c691-43ff-8a58-0972ff603f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:03.214775 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23ab76-c691-43ff-8a58-0972ff603f2d admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:03.215056 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23ab76-c691-43ff-8a58-0972ff603f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:03.215328 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23ab76-c691-43ff-8a58-0972ff603f2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:03.215981 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23ab76-c691-43ff-8a58-0972ff603f2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:03.217605 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23ab76-c691-43ff-8a58-0972ff603f2d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:03.232518 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 940/1878] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:03.233812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b20af-fbc9-4946-8b83-c4695db3463f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:03.252936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b20af-fbc9-4946-8b83-c4695db3463f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:03.272114 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f241e-3821-4a96-a20d-88c1ddbfe526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:03.272114 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f241e-3821-4a96-a20d-88c1ddbfe526 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:03.272114 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f241e-3821-4a96-a20d-88c1ddbfe526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:03.272114 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f241e-3821-4a96-a20d-88c1ddbfe526 admin admin] PATH_INFO: `/v3/projects/4cc2c5dc575a42e3baff0668c605caa0/users/f7afd87eda944d7fb608d2c853185b97/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:03.293632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f241e-3821-4a96-a20d-88c1ddbfe526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cc2c5dc575a42e3baff0668c605caa0, user_id=f7afd87eda944d7fb608d2c853185b97, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:03.296753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f241e-3821-4a96-a20d-88c1ddbfe526 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:03.326292 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 939/1879] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:03 2022] PUT /identity/v3/projects/4cc2c5dc575a42e3baff0668c605caa0/users/f7afd87eda944d7fb608d2c853185b97/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:03.328091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23ab76-c691-43ff-8a58-0972ff603f2d admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:03.340906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23ab76-c691-43ff-8a58-0972ff603f2d admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:03.431295 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa2668-ad53-4c3a-97cc-aab3897d74b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:03.432117 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa2668-ad53-4c3a-97cc-aab3897d74b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:03.432631 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa2668-ad53-4c3a-97cc-aab3897d74b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:03.433058 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa2668-ad53-4c3a-97cc-aab3897d74b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:03.434158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa2668-ad53-4c3a-97cc-aab3897d74b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:03.435957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa2668-ad53-4c3a-97cc-aab3897d74b4 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:03.451445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 941/1880] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:03.455098 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f241e-3821-4a96-a20d-88c1ddbfe526 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:03.467218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f241e-3821-4a96-a20d-88c1ddbfe526 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:03.480198 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5aa32-5756-48eb-979f-41e15e3e61af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:03.481155 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5aa32-5756-48eb-979f-41e15e3e61af admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:03.481796 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5aa32-5756-48eb-979f-41e15e3e61af admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:03.482482 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5aa32-5756-48eb-979f-41e15e3e61af admin admin] PATH_INFO: `/v3/projects/4cc2c5dc575a42e3baff0668c605caa0/users/f7afd87eda944d7fb608d2c853185b97/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:03.485274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5aa32-5756-48eb-979f-41e15e3e61af admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cc2c5dc575a42e3baff0668c605caa0, user_id=f7afd87eda944d7fb608d2c853185b97, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:03.489864 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5aa32-5756-48eb-979f-41e15e3e61af admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:03.512529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 940/1881] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:03 2022] PUT /identity/v3/projects/4cc2c5dc575a42e3baff0668c605caa0/users/f7afd87eda944d7fb608d2c853185b97/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:04.814615 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9d3a8-9ae7-430f-9503-719adc47efac None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:04.816972 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9d3a8-9ae7-430f-9503-719adc47efac None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:04.817488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9d3a8-9ae7-430f-9503-719adc47efac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:04.883393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-25a9d3a8-9ae7-430f-9503-719adc47efac None None] MFA Rules not processed for user `f7afd87eda944d7fb608d2c853185b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:04.887515 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-25a9d3a8-9ae7-430f-9503-719adc47efac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:05.049726 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:05.200982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 942/1882] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:05.214165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5aa32-5756-48eb-979f-41e15e3e61af admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:05.407900 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5aa32-5756-48eb-979f-41e15e3e61af admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:05.413377 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca5119-32b0-4e5f-99a5-d877fba88008 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:05.414420 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca5119-32b0-4e5f-99a5-d877fba88008 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:05.414866 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca5119-32b0-4e5f-99a5-d877fba88008 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:05.415280 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca5119-32b0-4e5f-99a5-d877fba88008 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:05.528078 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca5119-32b0-4e5f-99a5-d877fba88008 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:05.529810 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca5119-32b0-4e5f-99a5-d877fba88008 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:05.555364 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 941/1883] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:06.683092 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9d3a8-9ae7-430f-9503-719adc47efac None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:06.685304 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-25a9d3a8-9ae7-430f-9503-719adc47efac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:06.785218 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9d3a8-9ae7-430f-9503-719adc47efac None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:06.791077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccd0c5-70ee-4095-880c-4204822ac3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:06.791856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccd0c5-70ee-4095-880c-4204822ac3e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:06.792520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccd0c5-70ee-4095-880c-4204822ac3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:06.793023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccd0c5-70ee-4095-880c-4204822ac3e0 admin admin] PATH_INFO: `/v3/users/a2d0a5cfd07c437c8194cd89282f0715` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:06.794661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ccd0c5-70ee-4095-880c-4204822ac3e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2d0a5cfd07c437c8194cd89282f0715)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:06.797211 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ccd0c5-70ee-4095-880c-4204822ac3e0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:06.868509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-91ccd0c5-70ee-4095-880c-4204822ac3e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2d0a5cfd07c437c8194cd89282f0715'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:06.877790 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-91ccd0c5-70ee-4095-880c-4204822ac3e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2d0a5cfd07c437c8194cd89282f0715'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:06.896495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-91ccd0c5-70ee-4095-880c-4204822ac3e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2d0a5cfd07c437c8194cd89282f0715'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:06.898003 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-91ccd0c5-70ee-4095-880c-4204822ac3e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2d0a5cfd07c437c8194cd89282f0715'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:06.899498 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-91ccd0c5-70ee-4095-880c-4204822ac3e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2d0a5cfd07c437c8194cd89282f0715'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:06.927778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 943/1884] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:06 2022] DELETE /identity/v3/users/a2d0a5cfd07c437c8194cd89282f0715 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:06.933783 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b9720-e265-4f7a-a582-cf2f059de273 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:06.934389 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b9720-e265-4f7a-a582-cf2f059de273 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:06.934719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b9720-e265-4f7a-a582-cf2f059de273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:06.959798 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a33b9720-e265-4f7a-a582-cf2f059de273 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:06.960615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a33b9720-e265-4f7a-a582-cf2f059de273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:06.966048 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccd0c5-70ee-4095-880c-4204822ac3e0 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:06.967839 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-91ccd0c5-70ee-4095-880c-4204822ac3e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:06.986599 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:07.047830 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 942/1885] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:07.052080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:07.057537 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccd0c5-70ee-4095-880c-4204822ac3e0 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:07.066749 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5956956-f6c3-4c5e-8cbe-4df4603807da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:07.068387 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b9720-e265-4f7a-a582-cf2f059de273 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:07.068675 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5956956-f6c3-4c5e-8cbe-4df4603807da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:07.069116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5956956-f6c3-4c5e-8cbe-4df4603807da admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:07.069544 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5956956-f6c3-4c5e-8cbe-4df4603807da admin admin] PATH_INFO: `/v3/users/622831a733034b1a91ae3787eab450e1` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:07.069783 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a33b9720-e265-4f7a-a582-cf2f059de273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:07.071117 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5956956-f6c3-4c5e-8cbe-4df4603807da admin admin] RBAC: Authorizing `identity:delete_user(user_id=622831a733034b1a91ae3787eab450e1)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:07.075202 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5956956-f6c3-4c5e-8cbe-4df4603807da admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:07.146479 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b5956956-f6c3-4c5e-8cbe-4df4603807da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '622831a733034b1a91ae3787eab450e1'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.164855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b5956956-f6c3-4c5e-8cbe-4df4603807da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '622831a733034b1a91ae3787eab450e1'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.167262 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b9720-e265-4f7a-a582-cf2f059de273 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:07.175389 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1872a8f3-60e8-44a9-8576-02dc091e6fc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:07.176036 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1872a8f3-60e8-44a9-8576-02dc091e6fc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:07.177112 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1872a8f3-60e8-44a9-8576-02dc091e6fc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:07.177391 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1872a8f3-60e8-44a9-8576-02dc091e6fc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:07.189467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b5956956-f6c3-4c5e-8cbe-4df4603807da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '622831a733034b1a91ae3787eab450e1'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.192530 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b5956956-f6c3-4c5e-8cbe-4df4603807da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '622831a733034b1a91ae3787eab450e1'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.193328 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b5956956-f6c3-4c5e-8cbe-4df4603807da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '622831a733034b1a91ae3787eab450e1'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.194780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1872a8f3-60e8-44a9-8576-02dc091e6fc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:07.196684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1872a8f3-60e8-44a9-8576-02dc091e6fc6 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:07.220285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 944/1886] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:06 2022] DELETE /identity/v3/users/622831a733034b1a91ae3787eab450e1 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:07.224700 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425622e-6a57-487d-ad78-33af75a062b6 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:07.224934 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425622e-6a57-487d-ad78-33af75a062b6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:07.225195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425622e-6a57-487d-ad78-33af75a062b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:07.230555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 943/1887] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:07.246192 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-1425622e-6a57-487d-ad78-33af75a062b6 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:07.247452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1425622e-6a57-487d-ad78-33af75a062b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:07.318089 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 945/1888] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:07.331792 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1872a8f3-60e8-44a9-8576-02dc091e6fc6 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:07.341325 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1872a8f3-60e8-44a9-8576-02dc091e6fc6 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:07.414313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50663e43-a929-4b2a-8071-a4b5cc529e5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:07.414923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50663e43-a929-4b2a-8071-a4b5cc529e5b service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:07.415195 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50663e43-a929-4b2a-8071-a4b5cc529e5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:07.416100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50663e43-a929-4b2a-8071-a4b5cc529e5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:07.426620 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50663e43-a929-4b2a-8071-a4b5cc529e5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:07.428061 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50663e43-a929-4b2a-8071-a4b5cc529e5b service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:07.437984 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 944/1889] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:26:07.521454 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425622e-6a57-487d-ad78-33af75a062b6 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:07.522962 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1425622e-6a57-487d-ad78-33af75a062b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:07.542440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425622e-6a57-487d-ad78-33af75a062b6 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:07.548288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a3d46-ed02-46f4-8d5b-374f316f4303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:07.549017 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a3d46-ed02-46f4-8d5b-374f316f4303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:07.549455 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a3d46-ed02-46f4-8d5b-374f316f4303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:07.549628 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a3d46-ed02-46f4-8d5b-374f316f4303 admin admin] PATH_INFO: `/v3/projects/946af5bb215c43869d3cf63210cc8940` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:07.551024 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a3d46-ed02-46f4-8d5b-374f316f4303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=946af5bb215c43869d3cf63210cc8940)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:07.553194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a3d46-ed02-46f4-8d5b-374f316f4303 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:07.606856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ff9a3d46-ed02-46f4-8d5b-374f316f4303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '946af5bb215c43869d3cf63210cc8940', 'request_id': 'req-ff9a3d46-ed02-46f4-8d5b-374f316f4303'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.614935 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ff9a3d46-ed02-46f4-8d5b-374f316f4303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '946af5bb215c43869d3cf63210cc8940', 'request_id': 'req-ff9a3d46-ed02-46f4-8d5b-374f316f4303'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.625232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ff9a3d46-ed02-46f4-8d5b-374f316f4303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '946af5bb215c43869d3cf63210cc8940', 'request_id': 'req-ff9a3d46-ed02-46f4-8d5b-374f316f4303'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.639576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:07.645478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ff9a3d46-ed02-46f4-8d5b-374f316f4303 admin admin] The token cache is being invalidate because project 946af5bb215c43869d3cf63210cc8940 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:07.646180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ff9a3d46-ed02-46f4-8d5b-374f316f4303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.647911 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ff9a3d46-ed02-46f4-8d5b-374f316f4303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.652247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 946/1890] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:07 2022] DELETE /identity/v3/projects/946af5bb215c43869d3cf63210cc8940 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:07.656664 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50663e43-a929-4b2a-8071-a4b5cc529e5b service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:07.672848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50663e43-a929-4b2a-8071-a4b5cc529e5b service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:07.733313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54bcba5-6565-40ea-8e07-23d81ae141f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:07.734012 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bcba5-6565-40ea-8e07-23d81ae141f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:07.734434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bcba5-6565-40ea-8e07-23d81ae141f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:07.734773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bcba5-6565-40ea-8e07-23d81ae141f7 admin admin] PATH_INFO: `/v3/users/4462e8c191554902b494d275c38cf97a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:07.735990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54bcba5-6565-40ea-8e07-23d81ae141f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4462e8c191554902b494d275c38cf97a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:07.737989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54bcba5-6565-40ea-8e07-23d81ae141f7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:07.742883 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:07.787619 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f54bcba5-6565-40ea-8e07-23d81ae141f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4462e8c191554902b494d275c38cf97a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.795899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f54bcba5-6565-40ea-8e07-23d81ae141f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4462e8c191554902b494d275c38cf97a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.803532 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f54bcba5-6565-40ea-8e07-23d81ae141f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4462e8c191554902b494d275c38cf97a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.804218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f54bcba5-6565-40ea-8e07-23d81ae141f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4462e8c191554902b494d275c38cf97a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.804825 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f54bcba5-6565-40ea-8e07-23d81ae141f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4462e8c191554902b494d275c38cf97a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.825672 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 945/1891] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:07 2022] DELETE /identity/v3/users/4462e8c191554902b494d275c38cf97a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:07.874941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a3d46-ed02-46f4-8d5b-374f316f4303 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:07.876538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ff9a3d46-ed02-46f4-8d5b-374f316f4303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:07.890827 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a3d46-ed02-46f4-8d5b-374f316f4303 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:07.955183 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d749e-5fa3-4544-9b6e-349a67021abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:07.955822 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d749e-5fa3-4544-9b6e-349a67021abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:07.956104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d749e-5fa3-4544-9b6e-349a67021abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:07.956377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d749e-5fa3-4544-9b6e-349a67021abf admin admin] PATH_INFO: `/v3/projects/df806e6b8bc14a5dab778572144ca71b` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:07.957508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d749e-5fa3-4544-9b6e-349a67021abf admin admin] RBAC: Authorizing `identity:delete_project(project_id=df806e6b8bc14a5dab778572144ca71b)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:07.959465 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d749e-5fa3-4544-9b6e-349a67021abf admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:08.025991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-951d749e-5fa3-4544-9b6e-349a67021abf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df806e6b8bc14a5dab778572144ca71b', 'request_id': 'req-951d749e-5fa3-4544-9b6e-349a67021abf'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.032237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-951d749e-5fa3-4544-9b6e-349a67021abf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df806e6b8bc14a5dab778572144ca71b', 'request_id': 'req-951d749e-5fa3-4544-9b6e-349a67021abf'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.040295 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-951d749e-5fa3-4544-9b6e-349a67021abf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df806e6b8bc14a5dab778572144ca71b', 'request_id': 'req-951d749e-5fa3-4544-9b6e-349a67021abf'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.052851 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-951d749e-5fa3-4544-9b6e-349a67021abf admin admin] The token cache is being invalidate because project df806e6b8bc14a5dab778572144ca71b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:08.053145 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-951d749e-5fa3-4544-9b6e-349a67021abf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.053800 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-951d749e-5fa3-4544-9b6e-349a67021abf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.055344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 947/1892] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:07 2022] DELETE /identity/v3/projects/df806e6b8bc14a5dab778572144ca71b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:08.059876 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54bcba5-6565-40ea-8e07-23d81ae141f7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:08.061829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f54bcba5-6565-40ea-8e07-23d81ae141f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:08.075649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54bcba5-6565-40ea-8e07-23d81ae141f7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:08.127820 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d749e-5fa3-4544-9b6e-349a67021abf admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:08.129749 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-951d749e-5fa3-4544-9b6e-349a67021abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:08.140799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492788e8-a1b6-4f51-8b2c-b6a818ef23f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:08.141441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492788e8-a1b6-4f51-8b2c-b6a818ef23f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.141714 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492788e8-a1b6-4f51-8b2c-b6a818ef23f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.142048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492788e8-a1b6-4f51-8b2c-b6a818ef23f5 admin admin] PATH_INFO: `/v3/users/e7caeead95e143e9b8cfe257101e6d00` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.143274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492788e8-a1b6-4f51-8b2c-b6a818ef23f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7caeead95e143e9b8cfe257101e6d00)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:08.145531 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492788e8-a1b6-4f51-8b2c-b6a818ef23f5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:08.200731 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-492788e8-a1b6-4f51-8b2c-b6a818ef23f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7caeead95e143e9b8cfe257101e6d00'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.211320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-492788e8-a1b6-4f51-8b2c-b6a818ef23f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7caeead95e143e9b8cfe257101e6d00'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.219551 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-492788e8-a1b6-4f51-8b2c-b6a818ef23f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7caeead95e143e9b8cfe257101e6d00'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.220366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-492788e8-a1b6-4f51-8b2c-b6a818ef23f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7caeead95e143e9b8cfe257101e6d00'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.220545 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d749e-5fa3-4544-9b6e-349a67021abf admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:08.221001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-492788e8-a1b6-4f51-8b2c-b6a818ef23f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7caeead95e143e9b8cfe257101e6d00'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.227791 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978c7c4-0cb4-4bab-81f7-a51061f6f3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:08.228484 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978c7c4-0cb4-4bab-81f7-a51061f6f3d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.228764 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978c7c4-0cb4-4bab-81f7-a51061f6f3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.229037 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978c7c4-0cb4-4bab-81f7-a51061f6f3d0 admin admin] PATH_INFO: `/v3/projects/43dc70d1ed8b4dc5a682309f4c44a0c0` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.230273 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978c7c4-0cb4-4bab-81f7-a51061f6f3d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43dc70d1ed8b4dc5a682309f4c44a0c0)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:08.232232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978c7c4-0cb4-4bab-81f7-a51061f6f3d0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:08.246491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 946/1893] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:08 2022] DELETE /identity/v3/users/e7caeead95e143e9b8cfe257101e6d00 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:08.304427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:08.324745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9978c7c4-0cb4-4bab-81f7-a51061f6f3d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43dc70d1ed8b4dc5a682309f4c44a0c0', 'request_id': 'req-9978c7c4-0cb4-4bab-81f7-a51061f6f3d0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.331291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9978c7c4-0cb4-4bab-81f7-a51061f6f3d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43dc70d1ed8b4dc5a682309f4c44a0c0', 'request_id': 'req-9978c7c4-0cb4-4bab-81f7-a51061f6f3d0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.340593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9978c7c4-0cb4-4bab-81f7-a51061f6f3d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43dc70d1ed8b4dc5a682309f4c44a0c0', 'request_id': 'req-9978c7c4-0cb4-4bab-81f7-a51061f6f3d0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.354915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9978c7c4-0cb4-4bab-81f7-a51061f6f3d0 admin admin] The token cache is being invalidate because project 43dc70d1ed8b4dc5a682309f4c44a0c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:08.355208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9978c7c4-0cb4-4bab-81f7-a51061f6f3d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.356058 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9978c7c4-0cb4-4bab-81f7-a51061f6f3d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.357721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 948/1894] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:08 2022] DELETE /identity/v3/projects/43dc70d1ed8b4dc5a682309f4c44a0c0 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:08.362770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef527f-0623-4fd9-8229-e57f58cc54b9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.363058 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef527f-0623-4fd9-8229-e57f58cc54b9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.363329 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef527f-0623-4fd9-8229-e57f58cc54b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.372475 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:08.393334 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f5ef527f-0623-4fd9-8229-e57f58cc54b9 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:08.394760 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f5ef527f-0623-4fd9-8229-e57f58cc54b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:08.480830 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 947/1895] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:08.489123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0ab89-51b9-4e7a-ae01-deedac6e6a7f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.489499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0ab89-51b9-4e7a-ae01-deedac6e6a7f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.489913 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0ab89-51b9-4e7a-ae01-deedac6e6a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.513180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a8a0ab89-51b9-4e7a-ae01-deedac6e6a7f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:08.514741 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a8a0ab89-51b9-4e7a-ae01-deedac6e6a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:08.526240 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 949/1896] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:08.530700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef527f-0623-4fd9-8229-e57f58cc54b9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:08.542775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef527f-0623-4fd9-8229-e57f58cc54b9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:08.547866 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae3c64-d2fa-4ae5-8b0c-abd00201cada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:08.548559 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae3c64-d2fa-4ae5-8b0c-abd00201cada admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.548834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae3c64-d2fa-4ae5-8b0c-abd00201cada admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.549131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae3c64-d2fa-4ae5-8b0c-abd00201cada admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.549779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae3c64-d2fa-4ae5-8b0c-abd00201cada admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:08.550079 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae3c64-d2fa-4ae5-8b0c-abd00201cada admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:08.551855 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae3c64-d2fa-4ae5-8b0c-abd00201cada admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:08.569198 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 948/1897] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:26:08.570397 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0ab89-51b9-4e7a-ae01-deedac6e6a7f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:08.579844 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0ab89-51b9-4e7a-ae01-deedac6e6a7f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:08.584499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a5910-f642-4187-ba71-788f7a9d7ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:08.585036 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a5910-f642-4187-ba71-788f7a9d7ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.585314 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a5910-f642-4187-ba71-788f7a9d7ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.585584 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a5910-f642-4187-ba71-788f7a9d7ce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.586212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a5910-f642-4187-ba71-788f7a9d7ce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:08.588182 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a5910-f642-4187-ba71-788f7a9d7ce9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:08.624743 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae3c64-d2fa-4ae5-8b0c-abd00201cada admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:08.626717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-05ae3c64-d2fa-4ae5-8b0c-abd00201cada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:08.632083 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 950/1898] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:08 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:08.636726 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a5910-f642-4187-ba71-788f7a9d7ce9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:08.647251 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae3c64-d2fa-4ae5-8b0c-abd00201cada admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:08.708212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a5910-f642-4187-ba71-788f7a9d7ce9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:08.713738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7c04e-43d1-4d11-8931-b89d6960015f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:08.714381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7c04e-43d1-4d11-8931-b89d6960015f admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.714667 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7c04e-43d1-4d11-8931-b89d6960015f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.714987 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7c04e-43d1-4d11-8931-b89d6960015f admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.715692 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7c04e-43d1-4d11-8931-b89d6960015f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:08.717610 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7c04e-43d1-4d11-8931-b89d6960015f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:08.720696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6dd29-ab17-440b-b6ef-f38cf8b0cf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:08.721539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6dd29-ab17-440b-b6ef-f38cf8b0cf1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.721915 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6dd29-ab17-440b-b6ef-f38cf8b0cf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.722313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6dd29-ab17-440b-b6ef-f38cf8b0cf1c admin admin] PATH_INFO: `/v3/projects/b90e999421f54753a58707e43d672bde` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.723547 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6dd29-ab17-440b-b6ef-f38cf8b0cf1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b90e999421f54753a58707e43d672bde)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:08.725544 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6dd29-ab17-440b-b6ef-f38cf8b0cf1c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:08.747474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 951/1899] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:08 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:08.748660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7c04e-43d1-4d11-8931-b89d6960015f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:08.760275 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7c04e-43d1-4d11-8931-b89d6960015f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:08.766743 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349f647-d9d0-4061-b773-835bafce6348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:08.767901 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349f647-d9d0-4061-b773-835bafce6348 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.768189 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349f647-d9d0-4061-b773-835bafce6348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.768468 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349f647-d9d0-4061-b773-835bafce6348 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.769113 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349f647-d9d0-4061-b773-835bafce6348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:08.770745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349f647-d9d0-4061-b773-835bafce6348 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:08.781877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 952/1900] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:08.785456 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349f647-d9d0-4061-b773-835bafce6348 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:08.797775 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349f647-d9d0-4061-b773-835bafce6348 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:08.798810 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e8c6dd29-ab17-440b-b6ef-f38cf8b0cf1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b90e999421f54753a58707e43d672bde', 'request_id': 'req-e8c6dd29-ab17-440b-b6ef-f38cf8b0cf1c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.804359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e8c6dd29-ab17-440b-b6ef-f38cf8b0cf1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b90e999421f54753a58707e43d672bde', 'request_id': 'req-e8c6dd29-ab17-440b-b6ef-f38cf8b0cf1c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.812060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e8c6dd29-ab17-440b-b6ef-f38cf8b0cf1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b90e999421f54753a58707e43d672bde', 'request_id': 'req-e8c6dd29-ab17-440b-b6ef-f38cf8b0cf1c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.824253 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e8c6dd29-ab17-440b-b6ef-f38cf8b0cf1c admin admin] The token cache is being invalidate because project b90e999421f54753a58707e43d672bde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:08.824542 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e8c6dd29-ab17-440b-b6ef-f38cf8b0cf1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.825200 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e8c6dd29-ab17-440b-b6ef-f38cf8b0cf1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.826743 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 949/1901] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:08 2022] DELETE /identity/v3/projects/b90e999421f54753a58707e43d672bde => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:08.831223 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3a51a-63b6-45a6-a6c3-2a9046e4fc61 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.831510 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3a51a-63b6-45a6-a6c3-2a9046e4fc61 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.831781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3a51a-63b6-45a6-a6c3-2a9046e4fc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.851844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-00d3a51a-63b6-45a6-a6c3-2a9046e4fc61 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:08.853756 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-00d3a51a-63b6-45a6-a6c3-2a9046e4fc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:08.860210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07e680-1ae7-4b6e-bd04-5c81108f2ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:08.860822 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07e680-1ae7-4b6e-bd04-5c81108f2ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.861099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07e680-1ae7-4b6e-bd04-5c81108f2ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.861372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07e680-1ae7-4b6e-bd04-5c81108f2ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.862357 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07e680-1ae7-4b6e-bd04-5c81108f2ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:08.863934 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07e680-1ae7-4b6e-bd04-5c81108f2ee9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:08.870161 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:08.878183 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 953/1902] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:08.879354 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07e680-1ae7-4b6e-bd04-5c81108f2ee9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:08.882634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1b07e680-1ae7-4b6e-bd04-5c81108f2ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:08.891331 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:08.896820 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07e680-1ae7-4b6e-bd04-5c81108f2ee9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:08.904101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2fa4d-af09-40cf-93a7-15d1cbb29cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:08.905313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2fa4d-af09-40cf-93a7-15d1cbb29cfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.905696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2fa4d-af09-40cf-93a7-15d1cbb29cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.906393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2fa4d-af09-40cf-93a7-15d1cbb29cfb admin admin] PATH_INFO: `/v3/projects/c539d2a569f242e0a770d9dca45486a1/users/096010dba484470d96c8a6f76804db40/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.926145 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2fa4d-af09-40cf-93a7-15d1cbb29cfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c539d2a569f242e0a770d9dca45486a1, user_id=096010dba484470d96c8a6f76804db40, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:08.926565 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 950/1903] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:08.933523 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2fa4d-af09-40cf-93a7-15d1cbb29cfb admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:08.934672 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc5915-32aa-466d-8d21-d633d4d97e1b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.934954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc5915-32aa-466d-8d21-d633d4d97e1b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.935226 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc5915-32aa-466d-8d21-d633d4d97e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.949688 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 954/1904] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:08 2022] PUT /identity/v3/projects/c539d2a569f242e0a770d9dca45486a1/users/096010dba484470d96c8a6f76804db40/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:08.959881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-76bc5915-32aa-466d-8d21-d633d4d97e1b None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:08.961097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614f203-ad2a-4739-a39a-712b14555607 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.961390 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614f203-ad2a-4739-a39a-712b14555607 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.961523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-76bc5915-32aa-466d-8d21-d633d4d97e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:08.961691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614f203-ad2a-4739-a39a-712b14555607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.992744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6614f203-ad2a-4739-a39a-712b14555607 None None] MFA Rules not processed for user `096010dba484470d96c8a6f76804db40`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:08.994024 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6614f203-ad2a-4739-a39a-712b14555607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:09.032098 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 951/1905] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:09.036152 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc5915-32aa-466d-8d21-d633d4d97e1b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.045538 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc5915-32aa-466d-8d21-d633d4d97e1b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.050344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0492e71-bcd0-4f33-8fc3-766854c4c30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.050930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0492e71-bcd0-4f33-8fc3-766854c4c30e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.051212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0492e71-bcd0-4f33-8fc3-766854c4c30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.051479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0492e71-bcd0-4f33-8fc3-766854c4c30e admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.052115 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0492e71-bcd0-4f33-8fc3-766854c4c30e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:09.052417 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0492e71-bcd0-4f33-8fc3-766854c4c30e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.054033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0492e71-bcd0-4f33-8fc3-766854c4c30e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.063440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 955/1906] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:09.066512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 952/1907] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:26:09.068163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614f203-ad2a-4739-a39a-712b14555607 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.070560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0492e71-bcd0-4f33-8fc3-766854c4c30e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.078892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614f203-ad2a-4739-a39a-712b14555607 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.080800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0492e71-bcd0-4f33-8fc3-766854c4c30e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.083503 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e656103-8b37-4650-a38d-4a8148db3b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.084107 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e656103-8b37-4650-a38d-4a8148db3b63 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.084438 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e656103-8b37-4650-a38d-4a8148db3b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.084737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e656103-8b37-4650-a38d-4a8148db3b63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.085380 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e656103-8b37-4650-a38d-4a8148db3b63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.085493 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444d6b2-ad36-4b57-a9d6-bd8943bf98ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.086033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444d6b2-ad36-4b57-a9d6-bd8943bf98ec admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.086356 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444d6b2-ad36-4b57-a9d6-bd8943bf98ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.086632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444d6b2-ad36-4b57-a9d6-bd8943bf98ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.087271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444d6b2-ad36-4b57-a9d6-bd8943bf98ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.087364 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e656103-8b37-4650-a38d-4a8148db3b63 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.089182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444d6b2-ad36-4b57-a9d6-bd8943bf98ec admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.115443 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 956/1908] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:09 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:09.119570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 953/1909] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:09 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:09.120298 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e656103-8b37-4650-a38d-4a8148db3b63 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.120526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444d6b2-ad36-4b57-a9d6-bd8943bf98ec admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.186975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444d6b2-ad36-4b57-a9d6-bd8943bf98ec admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.190838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e656103-8b37-4650-a38d-4a8148db3b63 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.191915 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5882e7c-4912-415c-87e0-1194edb99130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.192518 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5882e7c-4912-415c-87e0-1194edb99130 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.192798 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5882e7c-4912-415c-87e0-1194edb99130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.193071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5882e7c-4912-415c-87e0-1194edb99130 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.193700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5882e7c-4912-415c-87e0-1194edb99130 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.195648 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5882e7c-4912-415c-87e0-1194edb99130 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.198226 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac413c-707a-4de5-9943-0cc029453861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.199217 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac413c-707a-4de5-9943-0cc029453861 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.199551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac413c-707a-4de5-9943-0cc029453861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.199827 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac413c-707a-4de5-9943-0cc029453861 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.200449 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac413c-707a-4de5-9943-0cc029453861 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.202379 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac413c-707a-4de5-9943-0cc029453861 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.219377 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 954/1910] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:09 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:09.223426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5882e7c-4912-415c-87e0-1194edb99130 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.227560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 957/1911] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:09 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:09.231823 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac413c-707a-4de5-9943-0cc029453861 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.233656 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5882e7c-4912-415c-87e0-1194edb99130 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.238181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae621f-70bc-450a-9bc6-c0ff4a546591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.238810 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae621f-70bc-450a-9bc6-c0ff4a546591 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.239083 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae621f-70bc-450a-9bc6-c0ff4a546591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.239346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae621f-70bc-450a-9bc6-c0ff4a546591 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.239980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae621f-70bc-450a-9bc6-c0ff4a546591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.241562 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae621f-70bc-450a-9bc6-c0ff4a546591 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.241820 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac413c-707a-4de5-9943-0cc029453861 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.247391 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db0605-0ef3-41dd-88a4-4376488a1de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.247975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db0605-0ef3-41dd-88a4-4376488a1de2 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.248251 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db0605-0ef3-41dd-88a4-4376488a1de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.248524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db0605-0ef3-41dd-88a4-4376488a1de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.249160 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db0605-0ef3-41dd-88a4-4376488a1de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.250769 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db0605-0ef3-41dd-88a4-4376488a1de2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.252171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 955/1912] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:09.255816 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae621f-70bc-450a-9bc6-c0ff4a546591 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.261490 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 958/1913] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:09.263799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:09.265413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db0605-0ef3-41dd-88a4-4376488a1de2 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.268007 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae621f-70bc-450a-9bc6-c0ff4a546591 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.273667 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd7de3-6ed0-4bdf-9b63-c93c507a0e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.274285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd7de3-6ed0-4bdf-9b63-c93c507a0e81 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.274617 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd7de3-6ed0-4bdf-9b63-c93c507a0e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.275440 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd7de3-6ed0-4bdf-9b63-c93c507a0e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.275925 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db0605-0ef3-41dd-88a4-4376488a1de2 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.276074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd7de3-6ed0-4bdf-9b63-c93c507a0e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.277657 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd7de3-6ed0-4bdf-9b63-c93c507a0e81 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.280481 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4ad1e-7b3d-4db5-9612-03233b1706df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.281054 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4ad1e-7b3d-4db5-9612-03233b1706df admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.281361 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4ad1e-7b3d-4db5-9612-03233b1706df admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.281632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4ad1e-7b3d-4db5-9612-03233b1706df admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.282278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4ad1e-7b3d-4db5-9612-03233b1706df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.283851 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4ad1e-7b3d-4db5-9612-03233b1706df admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.291017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 956/1914] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:09.294982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd7de3-6ed0-4bdf-9b63-c93c507a0e81 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.301244 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 959/1915] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:09.302425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4ad1e-7b3d-4db5-9612-03233b1706df admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.308835 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd7de3-6ed0-4bdf-9b63-c93c507a0e81 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.313779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7a0a1-d315-4669-8136-0c19fab768d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.314940 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7a0a1-d315-4669-8136-0c19fab768d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.315219 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7a0a1-d315-4669-8136-0c19fab768d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.316080 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7a0a1-d315-4669-8136-0c19fab768d8 admin admin] PATH_INFO: `/v3/projects/e2407311e1a84cbdb7dd50b5d64593d2/users/15cbf817499a4555870b71cc3f404b4c/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.321428 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4ad1e-7b3d-4db5-9612-03233b1706df admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.328752 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289ef1b-62f5-4fbc-a69f-26c2114a1a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.329506 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289ef1b-62f5-4fbc-a69f-26c2114a1a09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.329890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289ef1b-62f5-4fbc-a69f-26c2114a1a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.330284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289ef1b-62f5-4fbc-a69f-26c2114a1a09 admin admin] PATH_INFO: `/v3/projects/96811832fc194073b369f69c47703753/users/0b4c9f04a0c2459abcd2f0dd9c698e4a/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.331776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7a0a1-d315-4669-8136-0c19fab768d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2407311e1a84cbdb7dd50b5d64593d2, user_id=15cbf817499a4555870b71cc3f404b4c, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.335855 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7a0a1-d315-4669-8136-0c19fab768d8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.345821 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:09.349414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289ef1b-62f5-4fbc-a69f-26c2114a1a09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96811832fc194073b369f69c47703753, user_id=0b4c9f04a0c2459abcd2f0dd9c698e4a, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.352404 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289ef1b-62f5-4fbc-a69f-26c2114a1a09 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.355425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 957/1916] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:09 2022] PUT /identity/v3/projects/e2407311e1a84cbdb7dd50b5d64593d2/users/15cbf817499a4555870b71cc3f404b4c/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:09.369201 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 960/1917] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:09 2022] PUT /identity/v3/projects/96811832fc194073b369f69c47703753/users/0b4c9f04a0c2459abcd2f0dd9c698e4a/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:09.369745 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c6f8c-8a60-4a52-af5b-da0a05f9749f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.370041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c6f8c-8a60-4a52-af5b-da0a05f9749f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.370414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c6f8c-8a60-4a52-af5b-da0a05f9749f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.379978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97aa0b-56b3-4c25-8cbb-7e12d235b49b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.380261 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97aa0b-56b3-4c25-8cbb-7e12d235b49b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.380530 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97aa0b-56b3-4c25-8cbb-7e12d235b49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.397942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0f5c6f8c-8a60-4a52-af5b-da0a05f9749f None None] MFA Rules not processed for user `15cbf817499a4555870b71cc3f404b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:09.399217 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0f5c6f8c-8a60-4a52-af5b-da0a05f9749f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:09.408380 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-cf97aa0b-56b3-4c25-8cbb-7e12d235b49b None None] MFA Rules not processed for user `0b4c9f04a0c2459abcd2f0dd9c698e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:09.409556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cf97aa0b-56b3-4c25-8cbb-7e12d235b49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:09.474491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 958/1918] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:09.479031 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c6f8c-8a60-4a52-af5b-da0a05f9749f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.492008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 961/1919] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:09.497095 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97aa0b-56b3-4c25-8cbb-7e12d235b49b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.555397 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c6f8c-8a60-4a52-af5b-da0a05f9749f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.561490 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7961eb-ac65-46e5-98a2-9613b1fa6211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.562105 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7961eb-ac65-46e5-98a2-9613b1fa6211 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.562418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7961eb-ac65-46e5-98a2-9613b1fa6211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.562694 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7961eb-ac65-46e5-98a2-9613b1fa6211 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.563315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7961eb-ac65-46e5-98a2-9613b1fa6211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.563940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97aa0b-56b3-4c25-8cbb-7e12d235b49b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.565576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7961eb-ac65-46e5-98a2-9613b1fa6211 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.568933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e237c-0cf9-4945-8e4c-ef1e13001278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.574631 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e237c-0cf9-4945-8e4c-ef1e13001278 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.574930 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e237c-0cf9-4945-8e4c-ef1e13001278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.575219 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e237c-0cf9-4945-8e4c-ef1e13001278 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.575872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e237c-0cf9-4945-8e4c-ef1e13001278 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.578852 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e237c-0cf9-4945-8e4c-ef1e13001278 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.600534 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 959/1920] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:09 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:09.601726 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7961eb-ac65-46e5-98a2-9613b1fa6211 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.619413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 962/1921] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:09 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:09.623311 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e237c-0cf9-4945-8e4c-ef1e13001278 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.655168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:09.678194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7961eb-ac65-46e5-98a2-9613b1fa6211 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.706240 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e237c-0cf9-4945-8e4c-ef1e13001278 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.712284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95269e6-42e0-4a4c-8ff3-2c1fb8489930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.713257 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95269e6-42e0-4a4c-8ff3-2c1fb8489930 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.714031 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95269e6-42e0-4a4c-8ff3-2c1fb8489930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.714354 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95269e6-42e0-4a4c-8ff3-2c1fb8489930 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.715158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95269e6-42e0-4a4c-8ff3-2c1fb8489930 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.717093 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95269e6-42e0-4a4c-8ff3-2c1fb8489930 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.730975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:09.745557 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 963/1922] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:09 2022] POST /identity/v3/users => generated 295 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:09.749785 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95269e6-42e0-4a4c-8ff3-2c1fb8489930 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.750792 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048888c-5526-41d2-8fad-72fddd19015c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.751492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048888c-5526-41d2-8fad-72fddd19015c admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.751846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048888c-5526-41d2-8fad-72fddd19015c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.752187 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048888c-5526-41d2-8fad-72fddd19015c admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.752886 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048888c-5526-41d2-8fad-72fddd19015c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.755013 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048888c-5526-41d2-8fad-72fddd19015c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.761657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95269e6-42e0-4a4c-8ff3-2c1fb8489930 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.766755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d2c3f-a5a3-4d08-a56a-4d028602e609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.767430 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d2c3f-a5a3-4d08-a56a-4d028602e609 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.767818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d2c3f-a5a3-4d08-a56a-4d028602e609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.768154 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d2c3f-a5a3-4d08-a56a-4d028602e609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.768997 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d2c3f-a5a3-4d08-a56a-4d028602e609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.770683 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d2c3f-a5a3-4d08-a56a-4d028602e609 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.778672 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 960/1923] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:09 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:09.782061 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 964/1924] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:09.782617 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048888c-5526-41d2-8fad-72fddd19015c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.786000 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d2c3f-a5a3-4d08-a56a-4d028602e609 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.793265 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048888c-5526-41d2-8fad-72fddd19015c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.795993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d2c3f-a5a3-4d08-a56a-4d028602e609 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.797857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740419f-6184-403d-9f21-7403d5a634a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.798472 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740419f-6184-403d-9f21-7403d5a634a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.798751 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740419f-6184-403d-9f21-7403d5a634a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.799020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740419f-6184-403d-9f21-7403d5a634a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.799655 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740419f-6184-403d-9f21-7403d5a634a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.800580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5451f-44f4-4377-bd83-45a4b72e6565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.802594 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5451f-44f4-4377-bd83-45a4b72e6565 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.802727 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740419f-6184-403d-9f21-7403d5a634a0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.802815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5451f-44f4-4377-bd83-45a4b72e6565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.802815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5451f-44f4-4377-bd83-45a4b72e6565 admin admin] PATH_INFO: `/v3/projects/758409cca25048d587a5643a972d2e7c/users/a8096377a0904078ac3ace59b1e938e8/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.817936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5451f-44f4-4377-bd83-45a4b72e6565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=758409cca25048d587a5643a972d2e7c, user_id=a8096377a0904078ac3ace59b1e938e8, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.822027 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 961/1925] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:09.822172 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5451f-44f4-4377-bd83-45a4b72e6565 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.823412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740419f-6184-403d-9f21-7403d5a634a0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.834730 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740419f-6184-403d-9f21-7403d5a634a0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.838764 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 965/1926] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:09 2022] PUT /identity/v3/projects/758409cca25048d587a5643a972d2e7c/users/a8096377a0904078ac3ace59b1e938e8/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:09.842513 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5451f-44f4-4377-bd83-45a4b72e6565 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.905682 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82055de5-e988-41d2-81cb-fcb8540d6d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.906484 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82055de5-e988-41d2-81cb-fcb8540d6d52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.906965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82055de5-e988-41d2-81cb-fcb8540d6d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.907304 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82055de5-e988-41d2-81cb-fcb8540d6d52 admin admin] PATH_INFO: `/v3/projects/d2d189ec74704bbbb6b37849d17b4917/users/6feaa5c6f5954ee0b87875cd2c7f7052/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.910581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5451f-44f4-4377-bd83-45a4b72e6565 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.916318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837dd68-0e57-4962-834e-8b59f193c3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.916924 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837dd68-0e57-4962-834e-8b59f193c3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.917200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837dd68-0e57-4962-834e-8b59f193c3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.917593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837dd68-0e57-4962-834e-8b59f193c3f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.918347 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c837dd68-0e57-4962-834e-8b59f193c3f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.919883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c837dd68-0e57-4962-834e-8b59f193c3f7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.923878 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82055de5-e988-41d2-81cb-fcb8540d6d52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2d189ec74704bbbb6b37849d17b4917, user_id=6feaa5c6f5954ee0b87875cd2c7f7052, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.926778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82055de5-e988-41d2-81cb-fcb8540d6d52 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.934289 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 966/1927] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:09.937545 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837dd68-0e57-4962-834e-8b59f193c3f7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.945388 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 962/1928] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:09 2022] PUT /identity/v3/projects/d2d189ec74704bbbb6b37849d17b4917/users/6feaa5c6f5954ee0b87875cd2c7f7052/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:09.949261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82055de5-e988-41d2-81cb-fcb8540d6d52 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.950590 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837dd68-0e57-4962-834e-8b59f193c3f7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:10.013716 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f0440-ae9d-4ffd-a761-db052b7946d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:10.014409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f0440-ae9d-4ffd-a761-db052b7946d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.014690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f0440-ae9d-4ffd-a761-db052b7946d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.014964 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f0440-ae9d-4ffd-a761-db052b7946d2 admin admin] PATH_INFO: `/v3/projects/758409cca25048d587a5643a972d2e7c/users/a8096377a0904078ac3ace59b1e938e8/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.016876 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f0440-ae9d-4ffd-a761-db052b7946d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=758409cca25048d587a5643a972d2e7c, user_id=a8096377a0904078ac3ace59b1e938e8, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:10.019027 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:10.019549 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f0440-ae9d-4ffd-a761-db052b7946d2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:10.030614 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82055de5-e988-41d2-81cb-fcb8540d6d52 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:10.030910 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 967/1929] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:09 2022] PUT /identity/v3/projects/758409cca25048d587a5643a972d2e7c/users/a8096377a0904078ac3ace59b1e938e8/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:10.041571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf060b-f69d-4fda-97b0-d6f6be3b615b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.041855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf060b-f69d-4fda-97b0-d6f6be3b615b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.042158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf060b-f69d-4fda-97b0-d6f6be3b615b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.071237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-0ddf060b-f69d-4fda-97b0-d6f6be3b615b None None] MFA Rules not processed for user `a8096377a0904078ac3ace59b1e938e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:10.072507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0ddf060b-f69d-4fda-97b0-d6f6be3b615b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:10.097699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86675f1f-b7db-4028-92b9-474c71916ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:10.098381 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86675f1f-b7db-4028-92b9-474c71916ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.098620 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86675f1f-b7db-4028-92b9-474c71916ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.098890 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86675f1f-b7db-4028-92b9-474c71916ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.099528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86675f1f-b7db-4028-92b9-474c71916ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:10.101569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86675f1f-b7db-4028-92b9-474c71916ef4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:10.115837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 963/1930] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:10.117150 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86675f1f-b7db-4028-92b9-474c71916ef4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:10.123986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:10.128279 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86675f1f-b7db-4028-92b9-474c71916ef4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:10.133736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add13d6-e860-4aab-8ddd-48010d1788ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:10.134512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add13d6-e860-4aab-8ddd-48010d1788ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.134795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add13d6-e860-4aab-8ddd-48010d1788ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.135235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add13d6-e860-4aab-8ddd-48010d1788ed admin admin] PATH_INFO: `/v3/projects/d2d189ec74704bbbb6b37849d17b4917/users/6feaa5c6f5954ee0b87875cd2c7f7052/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.137776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add13d6-e860-4aab-8ddd-48010d1788ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2d189ec74704bbbb6b37849d17b4917, user_id=6feaa5c6f5954ee0b87875cd2c7f7052, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:10.140475 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add13d6-e860-4aab-8ddd-48010d1788ed admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:10.153016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 964/1931] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:10 2022] PUT /identity/v3/projects/d2d189ec74704bbbb6b37849d17b4917/users/6feaa5c6f5954ee0b87875cd2c7f7052/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:10.158359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 968/1932] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:10.163719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145fb5d-efc3-40be-9fbd-3c4a94131ec1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.164274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145fb5d-efc3-40be-9fbd-3c4a94131ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.164745 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145fb5d-efc3-40be-9fbd-3c4a94131ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.180337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf060b-f69d-4fda-97b0-d6f6be3b615b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:10.184300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0ddf060b-f69d-4fda-97b0-d6f6be3b615b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:10.203948 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7145fb5d-efc3-40be-9fbd-3c4a94131ec1 None None] MFA Rules not processed for user `6feaa5c6f5954ee0b87875cd2c7f7052`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:10.205187 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7145fb5d-efc3-40be-9fbd-3c4a94131ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:10.252871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf060b-f69d-4fda-97b0-d6f6be3b615b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:10.279496 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 965/1933] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:10.291731 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145fb5d-efc3-40be-9fbd-3c4a94131ec1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:10.293463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7145fb5d-efc3-40be-9fbd-3c4a94131ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:10.306152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780716c-31f3-4d69-8224-46ff20e5ae58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:10.306800 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780716c-31f3-4d69-8224-46ff20e5ae58 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.307763 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780716c-31f3-4d69-8224-46ff20e5ae58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.307763 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780716c-31f3-4d69-8224-46ff20e5ae58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.353758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145fb5d-efc3-40be-9fbd-3c4a94131ec1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:10.357530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ed64e-af15-4cb5-87bd-b2d10cf472e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:10.358134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ed64e-af15-4cb5-87bd-b2d10cf472e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.358445 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ed64e-af15-4cb5-87bd-b2d10cf472e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.358727 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ed64e-af15-4cb5-87bd-b2d10cf472e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.369070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ed64e-af15-4cb5-87bd-b2d10cf472e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:10.370551 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ed64e-af15-4cb5-87bd-b2d10cf472e1 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:10.371108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9780716c-31f3-4d69-8224-46ff20e5ae58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:10.372540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9780716c-31f3-4d69-8224-46ff20e5ae58 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:10.380276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 966/1934] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:26:10.381832 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 969/1935] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:10.451496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780716c-31f3-4d69-8224-46ff20e5ae58 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:10.465471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780716c-31f3-4d69-8224-46ff20e5ae58 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:10.470456 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ba748-e99b-4313-ac70-e2501c263c95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:10.471071 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ba748-e99b-4313-ac70-e2501c263c95 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.471353 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ba748-e99b-4313-ac70-e2501c263c95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.471627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ba748-e99b-4313-ac70-e2501c263c95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.517025 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:10.539111 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ba748-e99b-4313-ac70-e2501c263c95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:10.540875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ba748-e99b-4313-ac70-e2501c263c95 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:10.553054 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 970/1936] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:10.611968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ed64e-af15-4cb5-87bd-b2d10cf472e1 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:10.624643 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ed64e-af15-4cb5-87bd-b2d10cf472e1 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:10.629290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e4ba0-f2ce-40d0-b6e6-099e69677ce2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:10.630935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e4ba0-f2ce-40d0-b6e6-099e69677ce2 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.631398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e4ba0-f2ce-40d0-b6e6-099e69677ce2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.631991 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e4ba0-f2ce-40d0-b6e6-099e69677ce2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.642884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:10.713962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58e4ba0-f2ce-40d0-b6e6-099e69677ce2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:10.716011 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58e4ba0-f2ce-40d0-b6e6-099e69677ce2 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:10.729863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 967/1937] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:11.556584 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ba748-e99b-4313-ac70-e2501c263c95 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:11.566357 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ba748-e99b-4313-ac70-e2501c263c95 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:11.570792 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790638d0-8972-4efe-aef3-1461f0d9a578 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:11.571486 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790638d0-8972-4efe-aef3-1461f0d9a578 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:11.572330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790638d0-8972-4efe-aef3-1461f0d9a578 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:11.572611 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790638d0-8972-4efe-aef3-1461f0d9a578 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:11.574700 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-790638d0-8972-4efe-aef3-1461f0d9a578 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:11.650856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790638d0-8972-4efe-aef3-1461f0d9a578 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:11.652638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790638d0-8972-4efe-aef3-1461f0d9a578 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:11.675337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 971/1938] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:13.666212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e4ba0-f2ce-40d0-b6e6-099e69677ce2 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:13.737317 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e4ba0-f2ce-40d0-b6e6-099e69677ce2 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:13.742043 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a3a66-1b7e-403c-90aa-f891b7de1dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:13.742699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a3a66-1b7e-403c-90aa-f891b7de1dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:13.742972 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a3a66-1b7e-403c-90aa-f891b7de1dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:13.743240 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a3a66-1b7e-403c-90aa-f891b7de1dd9 admin admin] PATH_INFO: `/v3/users/15cbf817499a4555870b71cc3f404b4c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:13.744341 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a3a66-1b7e-403c-90aa-f891b7de1dd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15cbf817499a4555870b71cc3f404b4c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:13.746439 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a3a66-1b7e-403c-90aa-f891b7de1dd9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:13.793621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-377a3a66-1b7e-403c-90aa-f891b7de1dd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15cbf817499a4555870b71cc3f404b4c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:13.800902 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-377a3a66-1b7e-403c-90aa-f891b7de1dd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15cbf817499a4555870b71cc3f404b4c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:13.808064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-377a3a66-1b7e-403c-90aa-f891b7de1dd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15cbf817499a4555870b71cc3f404b4c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:13.808739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-377a3a66-1b7e-403c-90aa-f891b7de1dd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15cbf817499a4555870b71cc3f404b4c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:13.809346 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-377a3a66-1b7e-403c-90aa-f891b7de1dd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15cbf817499a4555870b71cc3f404b4c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:13.828556 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 968/1939] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:13 2022] DELETE /identity/v3/users/15cbf817499a4555870b71cc3f404b4c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:13.832899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fa4ae-b284-4d57-a317-8314335bd283 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:13.833351 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fa4ae-b284-4d57-a317-8314335bd283 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:13.833473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fa4ae-b284-4d57-a317-8314335bd283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:13.857378 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-420fa4ae-b284-4d57-a317-8314335bd283 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:13.858647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-420fa4ae-b284-4d57-a317-8314335bd283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:13.925850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 972/1940] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:13.932697 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a3a66-1b7e-403c-90aa-f891b7de1dd9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:13.934211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-377a3a66-1b7e-403c-90aa-f891b7de1dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:13.968966 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:14.000305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a3a66-1b7e-403c-90aa-f891b7de1dd9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:14.004079 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257a6a5-5ace-45f2-abcb-68b0eb1bec38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:14.004686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257a6a5-5ace-45f2-abcb-68b0eb1bec38 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:14.004969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257a6a5-5ace-45f2-abcb-68b0eb1bec38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:14.005246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257a6a5-5ace-45f2-abcb-68b0eb1bec38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:14.015313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257a6a5-5ace-45f2-abcb-68b0eb1bec38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:14.016934 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257a6a5-5ace-45f2-abcb-68b0eb1bec38 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:14.028646 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 969/1941] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:14.390524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420fa4ae-b284-4d57-a317-8314335bd283 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:14.391965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-420fa4ae-b284-4d57-a317-8314335bd283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:14.409075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420fa4ae-b284-4d57-a317-8314335bd283 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:14.430565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867460d-5e96-451b-a5e3-198ea990c1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:14.431849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867460d-5e96-451b-a5e3-198ea990c1b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:14.432122 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867460d-5e96-451b-a5e3-198ea990c1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:14.432391 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867460d-5e96-451b-a5e3-198ea990c1b5 admin admin] PATH_INFO: `/v3/projects/e2407311e1a84cbdb7dd50b5d64593d2` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:14.434087 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867460d-5e96-451b-a5e3-198ea990c1b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2407311e1a84cbdb7dd50b5d64593d2)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:14.435916 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867460d-5e96-451b-a5e3-198ea990c1b5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:14.500305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257a6a5-5ace-45f2-abcb-68b0eb1bec38 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:14.501879 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8257a6a5-5ace-45f2-abcb-68b0eb1bec38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:14.517923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257a6a5-5ace-45f2-abcb-68b0eb1bec38 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:14.547978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7867460d-5e96-451b-a5e3-198ea990c1b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2407311e1a84cbdb7dd50b5d64593d2', 'request_id': 'req-7867460d-5e96-451b-a5e3-198ea990c1b5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.558067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7867460d-5e96-451b-a5e3-198ea990c1b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2407311e1a84cbdb7dd50b5d64593d2', 'request_id': 'req-7867460d-5e96-451b-a5e3-198ea990c1b5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.574290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7867460d-5e96-451b-a5e3-198ea990c1b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2407311e1a84cbdb7dd50b5d64593d2', 'request_id': 'req-7867460d-5e96-451b-a5e3-198ea990c1b5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.583948 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:14.590286 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7867460d-5e96-451b-a5e3-198ea990c1b5 admin admin] The token cache is being invalidate because project e2407311e1a84cbdb7dd50b5d64593d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:14.590690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7867460d-5e96-451b-a5e3-198ea990c1b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.591826 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7867460d-5e96-451b-a5e3-198ea990c1b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.596970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 973/1942] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:14 2022] DELETE /identity/v3/projects/e2407311e1a84cbdb7dd50b5d64593d2 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:14.597706 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1cfd37-dc99-4dfe-856f-e19672bab3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:14.598018 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867460d-5e96-451b-a5e3-198ea990c1b5 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:14.598765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cfd37-dc99-4dfe-856f-e19672bab3c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:14.599323 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cfd37-dc99-4dfe-856f-e19672bab3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:14.599614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7867460d-5e96-451b-a5e3-198ea990c1b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:14.599750 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cfd37-dc99-4dfe-856f-e19672bab3c1 admin admin] PATH_INFO: `/v3/users/096010dba484470d96c8a6f76804db40` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:14.601866 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1cfd37-dc99-4dfe-856f-e19672bab3c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=096010dba484470d96c8a6f76804db40)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:14.603899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1cfd37-dc99-4dfe-856f-e19672bab3c1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:14.622559 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867460d-5e96-451b-a5e3-198ea990c1b5 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:14.630206 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b2cbf-12b8-43df-b370-8ab539c9381b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:14.630416 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b2cbf-12b8-43df-b370-8ab539c9381b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:14.630665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b2cbf-12b8-43df-b370-8ab539c9381b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:14.631006 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b2cbf-12b8-43df-b370-8ab539c9381b admin admin] PATH_INFO: `/v3/users/6feaa5c6f5954ee0b87875cd2c7f7052` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:14.632238 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b2cbf-12b8-43df-b370-8ab539c9381b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6feaa5c6f5954ee0b87875cd2c7f7052)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:14.635413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b2cbf-12b8-43df-b370-8ab539c9381b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:14.690945 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0c1cfd37-dc99-4dfe-856f-e19672bab3c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '096010dba484470d96c8a6f76804db40'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.703092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0c1cfd37-dc99-4dfe-856f-e19672bab3c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '096010dba484470d96c8a6f76804db40'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.705794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a11b2cbf-12b8-43df-b370-8ab539c9381b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6feaa5c6f5954ee0b87875cd2c7f7052'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.716309 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a11b2cbf-12b8-43df-b370-8ab539c9381b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6feaa5c6f5954ee0b87875cd2c7f7052'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.723361 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0c1cfd37-dc99-4dfe-856f-e19672bab3c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '096010dba484470d96c8a6f76804db40'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.724338 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0c1cfd37-dc99-4dfe-856f-e19672bab3c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '096010dba484470d96c8a6f76804db40'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.725145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0c1cfd37-dc99-4dfe-856f-e19672bab3c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '096010dba484470d96c8a6f76804db40'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.728456 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a11b2cbf-12b8-43df-b370-8ab539c9381b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6feaa5c6f5954ee0b87875cd2c7f7052'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.729467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a11b2cbf-12b8-43df-b370-8ab539c9381b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6feaa5c6f5954ee0b87875cd2c7f7052'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.730117 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a11b2cbf-12b8-43df-b370-8ab539c9381b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6feaa5c6f5954ee0b87875cd2c7f7052'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.756133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 970/1943] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:14 2022] DELETE /identity/v3/users/096010dba484470d96c8a6f76804db40 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:14.765027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 974/1944] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:14 2022] DELETE /identity/v3/users/6feaa5c6f5954ee0b87875cd2c7f7052 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:14.765199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b351f-da8f-4d1c-8f03-2c22220c5127 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:14.765278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b351f-da8f-4d1c-8f03-2c22220c5127 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:14.765547 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b351f-da8f-4d1c-8f03-2c22220c5127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:14.787240 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6d2b351f-da8f-4d1c-8f03-2c22220c5127 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:14.788487 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6d2b351f-da8f-4d1c-8f03-2c22220c5127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:14.841210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:14.862023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 971/1945] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:14.892993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b2cbf-12b8-43df-b370-8ab539c9381b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:14.894829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a11b2cbf-12b8-43df-b370-8ab539c9381b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:14.984078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b2cbf-12b8-43df-b370-8ab539c9381b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:14.988719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de8006-8d82-48be-97e3-96e1801c39ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:14.988719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de8006-8d82-48be-97e3-96e1801c39ba service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:14.988918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de8006-8d82-48be-97e3-96e1801c39ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:14.989477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de8006-8d82-48be-97e3-96e1801c39ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:15.000311 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de8006-8d82-48be-97e3-96e1801c39ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:15.001869 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de8006-8d82-48be-97e3-96e1801c39ba service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:15.013611 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 975/1946] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:26:15.288559 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b351f-da8f-4d1c-8f03-2c22220c5127 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:15.290018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6d2b351f-da8f-4d1c-8f03-2c22220c5127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:15.308997 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b351f-da8f-4d1c-8f03-2c22220c5127 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:15.323720 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b53aa3-aa77-4a79-ae6c-674cbcf4a489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:15.324383 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b53aa3-aa77-4a79-ae6c-674cbcf4a489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:15.324667 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b53aa3-aa77-4a79-ae6c-674cbcf4a489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:15.324938 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b53aa3-aa77-4a79-ae6c-674cbcf4a489 admin admin] PATH_INFO: `/v3/projects/d2d189ec74704bbbb6b37849d17b4917` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:15.326079 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b53aa3-aa77-4a79-ae6c-674cbcf4a489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2d189ec74704bbbb6b37849d17b4917)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:15.328047 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b53aa3-aa77-4a79-ae6c-674cbcf4a489 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:15.391140 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-50b53aa3-aa77-4a79-ae6c-674cbcf4a489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2d189ec74704bbbb6b37849d17b4917', 'request_id': 'req-50b53aa3-aa77-4a79-ae6c-674cbcf4a489'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:15.400321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-50b53aa3-aa77-4a79-ae6c-674cbcf4a489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2d189ec74704bbbb6b37849d17b4917', 'request_id': 'req-50b53aa3-aa77-4a79-ae6c-674cbcf4a489'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:15.410012 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-50b53aa3-aa77-4a79-ae6c-674cbcf4a489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2d189ec74704bbbb6b37849d17b4917', 'request_id': 'req-50b53aa3-aa77-4a79-ae6c-674cbcf4a489'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:15.425756 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-50b53aa3-aa77-4a79-ae6c-674cbcf4a489 admin admin] The token cache is being invalidate because project d2d189ec74704bbbb6b37849d17b4917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:15.425756 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-50b53aa3-aa77-4a79-ae6c-674cbcf4a489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:15.426424 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-50b53aa3-aa77-4a79-ae6c-674cbcf4a489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:15.428238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 972/1947] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:15 2022] DELETE /identity/v3/projects/d2d189ec74704bbbb6b37849d17b4917 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:15.433706 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286d993-56d4-4c26-910d-638800a98fdc None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:15.434072 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286d993-56d4-4c26-910d-638800a98fdc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:15.434489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286d993-56d4-4c26-910d-638800a98fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:15.467099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-0286d993-56d4-4c26-910d-638800a98fdc None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:15.469298 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0286d993-56d4-4c26-910d-638800a98fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:15.523582 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b53aa3-aa77-4a79-ae6c-674cbcf4a489 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:15.525890 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-50b53aa3-aa77-4a79-ae6c-674cbcf4a489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:15.601858 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 976/1948] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:15.609853 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf0253-3856-4003-9136-541a18f18914 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:15.610150 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf0253-3856-4003-9136-541a18f18914 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:15.610489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf0253-3856-4003-9136-541a18f18914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:15.644288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c5cf0253-3856-4003-9136-541a18f18914 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:15.645749 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c5cf0253-3856-4003-9136-541a18f18914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:15.658298 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 977/1949] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:15.660240 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf0253-3856-4003-9136-541a18f18914 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:15.672375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b53aa3-aa77-4a79-ae6c-674cbcf4a489 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:15.675381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf0253-3856-4003-9136-541a18f18914 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:15.678437 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2f3ed-2e9c-44ba-aefd-564c0f44f4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:15.684466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2f3ed-2e9c-44ba-aefd-564c0f44f4d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:15.685168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2f3ed-2e9c-44ba-aefd-564c0f44f4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:15.685901 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2f3ed-2e9c-44ba-aefd-564c0f44f4d3 admin admin] PATH_INFO: `/v3/projects/c539d2a569f242e0a770d9dca45486a1` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:15.688943 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2f3ed-2e9c-44ba-aefd-564c0f44f4d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c539d2a569f242e0a770d9dca45486a1)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:15.691309 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2f3ed-2e9c-44ba-aefd-564c0f44f4d3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:15.691908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905fc92-3c7e-49c9-aeae-ec1266d71fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:15.692156 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905fc92-3c7e-49c9-aeae-ec1266d71fdf admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:15.692417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905fc92-3c7e-49c9-aeae-ec1266d71fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:15.692824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905fc92-3c7e-49c9-aeae-ec1266d71fdf admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:15.693467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905fc92-3c7e-49c9-aeae-ec1266d71fdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:15.693770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905fc92-3c7e-49c9-aeae-ec1266d71fdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:15.696138 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905fc92-3c7e-49c9-aeae-ec1266d71fdf admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:15.698354 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:15.722717 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 978/1950] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:26:15.731353 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905fc92-3c7e-49c9-aeae-ec1266d71fdf admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:15.746813 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905fc92-3c7e-49c9-aeae-ec1266d71fdf admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:15.754574 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9cf43-0431-40ce-9557-5b24751a6c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:15.755175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9cf43-0431-40ce-9557-5b24751a6c0a admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:15.755648 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9cf43-0431-40ce-9557-5b24751a6c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:15.755830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9cf43-0431-40ce-9557-5b24751a6c0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:15.756564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9cf43-0431-40ce-9557-5b24751a6c0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:15.759890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9cf43-0431-40ce-9557-5b24751a6c0a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:15.776977 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-07f2f3ed-2e9c-44ba-aefd-564c0f44f4d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c539d2a569f242e0a770d9dca45486a1', 'request_id': 'req-07f2f3ed-2e9c-44ba-aefd-564c0f44f4d3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:15.784742 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-07f2f3ed-2e9c-44ba-aefd-564c0f44f4d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c539d2a569f242e0a770d9dca45486a1', 'request_id': 'req-07f2f3ed-2e9c-44ba-aefd-564c0f44f4d3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:15.796907 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-07f2f3ed-2e9c-44ba-aefd-564c0f44f4d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c539d2a569f242e0a770d9dca45486a1', 'request_id': 'req-07f2f3ed-2e9c-44ba-aefd-564c0f44f4d3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:15.804236 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 979/1951] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:15 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:15.807993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9cf43-0431-40ce-9557-5b24751a6c0a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:15.811808 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-07f2f3ed-2e9c-44ba-aefd-564c0f44f4d3 admin admin] The token cache is being invalidate because project c539d2a569f242e0a770d9dca45486a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:15.812174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-07f2f3ed-2e9c-44ba-aefd-564c0f44f4d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:15.813483 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-07f2f3ed-2e9c-44ba-aefd-564c0f44f4d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:15.815530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 973/1952] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:15 2022] DELETE /identity/v3/projects/c539d2a569f242e0a770d9dca45486a1 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:15.819812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2f3ed-2e9c-44ba-aefd-564c0f44f4d3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:15.822311 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-07f2f3ed-2e9c-44ba-aefd-564c0f44f4d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:15.833971 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:15.889733 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9cf43-0431-40ce-9557-5b24751a6c0a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:15.894954 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270ea01-1bd1-4859-b094-43884b9a14f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:15.896759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270ea01-1bd1-4859-b094-43884b9a14f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:15.897052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270ea01-1bd1-4859-b094-43884b9a14f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:15.897344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270ea01-1bd1-4859-b094-43884b9a14f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:15.898591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270ea01-1bd1-4859-b094-43884b9a14f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:15.902685 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270ea01-1bd1-4859-b094-43884b9a14f0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:15.933455 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 980/1953] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:15 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:15.937045 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2f3ed-2e9c-44ba-aefd-564c0f44f4d3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:15.938392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270ea01-1bd1-4859-b094-43884b9a14f0 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:15.940617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a270ea01-1bd1-4859-b094-43884b9a14f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:15.943067 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3acc8-fc87-4e3a-850f-4b0b448c1505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:15.943908 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3acc8-fc87-4e3a-850f-4b0b448c1505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:15.944415 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3acc8-fc87-4e3a-850f-4b0b448c1505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:15.944788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3acc8-fc87-4e3a-850f-4b0b448c1505 admin admin] PATH_INFO: `/v3/users/0b4c9f04a0c2459abcd2f0dd9c698e4a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:15.946171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3acc8-fc87-4e3a-850f-4b0b448c1505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b4c9f04a0c2459abcd2f0dd9c698e4a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:15.949040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3acc8-fc87-4e3a-850f-4b0b448c1505 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:15.960044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270ea01-1bd1-4859-b094-43884b9a14f0 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:15.968209 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19601728-be6d-4225-afd0-d3d08b425a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:15.969307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19601728-be6d-4225-afd0-d3d08b425a50 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:15.969591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19601728-be6d-4225-afd0-d3d08b425a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:15.969863 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19601728-be6d-4225-afd0-d3d08b425a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:15.970883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19601728-be6d-4225-afd0-d3d08b425a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:15.972645 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19601728-be6d-4225-afd0-d3d08b425a50 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:15.991300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 981/1954] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:15.995934 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19601728-be6d-4225-afd0-d3d08b425a50 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:16.007249 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19601728-be6d-4225-afd0-d3d08b425a50 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:16.013257 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49075209-f897-4272-9c68-4ea0e02ac28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:16.013854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49075209-f897-4272-9c68-4ea0e02ac28d admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:16.014135 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49075209-f897-4272-9c68-4ea0e02ac28d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:16.014444 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49075209-f897-4272-9c68-4ea0e02ac28d admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:16.015082 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49075209-f897-4272-9c68-4ea0e02ac28d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:16.016034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b9a3acc8-fc87-4e3a-850f-4b0b448c1505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b4c9f04a0c2459abcd2f0dd9c698e4a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.016704 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49075209-f897-4272-9c68-4ea0e02ac28d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:16.025440 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b9a3acc8-fc87-4e3a-850f-4b0b448c1505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b4c9f04a0c2459abcd2f0dd9c698e4a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.029172 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 982/1955] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:16.033200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49075209-f897-4272-9c68-4ea0e02ac28d admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:16.033659 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b9a3acc8-fc87-4e3a-850f-4b0b448c1505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b4c9f04a0c2459abcd2f0dd9c698e4a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.034397 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b9a3acc8-fc87-4e3a-850f-4b0b448c1505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b4c9f04a0c2459abcd2f0dd9c698e4a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.035020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b9a3acc8-fc87-4e3a-850f-4b0b448c1505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b4c9f04a0c2459abcd2f0dd9c698e4a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.044125 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49075209-f897-4272-9c68-4ea0e02ac28d admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:16.050085 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c67920-0d19-49bb-8911-1cdfdd1f39fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:16.050904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c67920-0d19-49bb-8911-1cdfdd1f39fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:16.051356 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c67920-0d19-49bb-8911-1cdfdd1f39fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:16.051641 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c67920-0d19-49bb-8911-1cdfdd1f39fc admin admin] PATH_INFO: `/v3/projects/a6c2b0e41cfa4ec298b81cf5261eba51/users/acb64119420049f690fa2ef02daf7212/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:16.059462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 974/1956] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:15 2022] DELETE /identity/v3/users/0b4c9f04a0c2459abcd2f0dd9c698e4a => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:16.083146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c67920-0d19-49bb-8911-1cdfdd1f39fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6c2b0e41cfa4ec298b81cf5261eba51, user_id=acb64119420049f690fa2ef02daf7212, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:16.086883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c67920-0d19-49bb-8911-1cdfdd1f39fc admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:16.119707 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 983/1957] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:16 2022] PUT /identity/v3/projects/a6c2b0e41cfa4ec298b81cf5261eba51/users/acb64119420049f690fa2ef02daf7212/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:16.121982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca79eb3-4c62-496d-a634-bb39aca86169 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:16.122343 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca79eb3-4c62-496d-a634-bb39aca86169 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:16.122714 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca79eb3-4c62-496d-a634-bb39aca86169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:16.160403 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5ca79eb3-4c62-496d-a634-bb39aca86169 None None] MFA Rules not processed for user `acb64119420049f690fa2ef02daf7212`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:16.162043 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5ca79eb3-4c62-496d-a634-bb39aca86169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:16.244102 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 975/1958] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:16.256602 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c67920-0d19-49bb-8911-1cdfdd1f39fc admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:16.258896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f9c67920-0d19-49bb-8911-1cdfdd1f39fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:16.336030 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c67920-0d19-49bb-8911-1cdfdd1f39fc admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:16.339726 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25435689-80d2-42d9-9344-10d9b62b7292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:16.340332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25435689-80d2-42d9-9344-10d9b62b7292 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:16.340610 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25435689-80d2-42d9-9344-10d9b62b7292 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:16.340879 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25435689-80d2-42d9-9344-10d9b62b7292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:16.349431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25435689-80d2-42d9-9344-10d9b62b7292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:16.350897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25435689-80d2-42d9-9344-10d9b62b7292 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:16.360773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 984/1959] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:16.383100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca79eb3-4c62-496d-a634-bb39aca86169 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:16.385098 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5ca79eb3-4c62-496d-a634-bb39aca86169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:16.399339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:16.463729 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca79eb3-4c62-496d-a634-bb39aca86169 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:16.465785 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25435689-80d2-42d9-9344-10d9b62b7292 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:16.467344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-25435689-80d2-42d9-9344-10d9b62b7292 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:16.470724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ea31c-f1d3-4eed-a9bc-09845b7df407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:16.471622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ea31c-f1d3-4eed-a9bc-09845b7df407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:16.472058 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ea31c-f1d3-4eed-a9bc-09845b7df407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:16.472525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ea31c-f1d3-4eed-a9bc-09845b7df407 admin admin] PATH_INFO: `/v3/users/acb64119420049f690fa2ef02daf7212` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:16.474113 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ea31c-f1d3-4eed-a9bc-09845b7df407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acb64119420049f690fa2ef02daf7212)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:16.476965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ea31c-f1d3-4eed-a9bc-09845b7df407 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:16.487775 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25435689-80d2-42d9-9344-10d9b62b7292 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:16.502598 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acaaf39-d5e4-40fe-b4a2-7b812902269a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:16.503250 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaaf39-d5e4-40fe-b4a2-7b812902269a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:16.503555 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaaf39-d5e4-40fe-b4a2-7b812902269a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:16.503777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaaf39-d5e4-40fe-b4a2-7b812902269a admin admin] PATH_INFO: `/v3/projects/96811832fc194073b369f69c47703753` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:16.505369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acaaf39-d5e4-40fe-b4a2-7b812902269a admin admin] RBAC: Authorizing `identity:delete_project(project_id=96811832fc194073b369f69c47703753)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:16.507429 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acaaf39-d5e4-40fe-b4a2-7b812902269a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:16.521286 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:16.559339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-170ea31c-f1d3-4eed-a9bc-09845b7df407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acb64119420049f690fa2ef02daf7212'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.570760 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9acaaf39-d5e4-40fe-b4a2-7b812902269a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96811832fc194073b369f69c47703753', 'request_id': 'req-9acaaf39-d5e4-40fe-b4a2-7b812902269a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.572113 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-170ea31c-f1d3-4eed-a9bc-09845b7df407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acb64119420049f690fa2ef02daf7212'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.577387 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9acaaf39-d5e4-40fe-b4a2-7b812902269a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96811832fc194073b369f69c47703753', 'request_id': 'req-9acaaf39-d5e4-40fe-b4a2-7b812902269a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.582581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-170ea31c-f1d3-4eed-a9bc-09845b7df407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acb64119420049f690fa2ef02daf7212'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.583336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-170ea31c-f1d3-4eed-a9bc-09845b7df407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acb64119420049f690fa2ef02daf7212'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.584198 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-170ea31c-f1d3-4eed-a9bc-09845b7df407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acb64119420049f690fa2ef02daf7212'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.586875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9acaaf39-d5e4-40fe-b4a2-7b812902269a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96811832fc194073b369f69c47703753', 'request_id': 'req-9acaaf39-d5e4-40fe-b4a2-7b812902269a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.600163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9acaaf39-d5e4-40fe-b4a2-7b812902269a admin admin] The token cache is being invalidate because project 96811832fc194073b369f69c47703753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:16.600466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9acaaf39-d5e4-40fe-b4a2-7b812902269a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.601173 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9acaaf39-d5e4-40fe-b4a2-7b812902269a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.604007 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 985/1960] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:16 2022] DELETE /identity/v3/projects/96811832fc194073b369f69c47703753 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:16.605621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 976/1961] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:16 2022] DELETE /identity/v3/users/acb64119420049f690fa2ef02daf7212 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:16.609508 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ea31c-f1d3-4eed-a9bc-09845b7df407 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:16.609907 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23720bf1-aea8-4cd2-b194-3ff0098dc47d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:16.610197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23720bf1-aea8-4cd2-b194-3ff0098dc47d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:16.610505 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23720bf1-aea8-4cd2-b194-3ff0098dc47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:16.611507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-170ea31c-f1d3-4eed-a9bc-09845b7df407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:16.630464 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-23720bf1-aea8-4cd2-b194-3ff0098dc47d None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:16.631687 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-23720bf1-aea8-4cd2-b194-3ff0098dc47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:16.705314 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ea31c-f1d3-4eed-a9bc-09845b7df407 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:16.709934 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 986/1962] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:16.710823 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc0723-638e-47e7-85bb-eed208b8b13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:16.711641 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc0723-638e-47e7-85bb-eed208b8b13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:16.711939 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc0723-638e-47e7-85bb-eed208b8b13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:16.712799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc0723-638e-47e7-85bb-eed208b8b13b admin admin] PATH_INFO: `/v3/users/a8096377a0904078ac3ace59b1e938e8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:16.714391 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc0723-638e-47e7-85bb-eed208b8b13b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8096377a0904078ac3ace59b1e938e8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:16.720741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc0723-638e-47e7-85bb-eed208b8b13b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:16.735257 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23720bf1-aea8-4cd2-b194-3ff0098dc47d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:16.740191 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-23720bf1-aea8-4cd2-b194-3ff0098dc47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:16.763341 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:16.780241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-39fc0723-638e-47e7-85bb-eed208b8b13b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8096377a0904078ac3ace59b1e938e8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.799853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-39fc0723-638e-47e7-85bb-eed208b8b13b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8096377a0904078ac3ace59b1e938e8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.834239 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-39fc0723-638e-47e7-85bb-eed208b8b13b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8096377a0904078ac3ace59b1e938e8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.834239 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-39fc0723-638e-47e7-85bb-eed208b8b13b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8096377a0904078ac3ace59b1e938e8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.834239 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-39fc0723-638e-47e7-85bb-eed208b8b13b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8096377a0904078ac3ace59b1e938e8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.835488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23720bf1-aea8-4cd2-b194-3ff0098dc47d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:16.841264 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b7508-36e8-432e-9b27-bd3a8ad543ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:16.842139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b7508-36e8-432e-9b27-bd3a8ad543ff service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:16.842444 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b7508-36e8-432e-9b27-bd3a8ad543ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:16.843147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b7508-36e8-432e-9b27-bd3a8ad543ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:16.845352 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-274b7508-36e8-432e-9b27-bd3a8ad543ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:16.862169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 977/1963] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:16 2022] DELETE /identity/v3/users/a8096377a0904078ac3ace59b1e938e8 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:16.865296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b7508-36e8-432e-9b27-bd3a8ad543ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:16.865296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b7508-36e8-432e-9b27-bd3a8ad543ff service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:16.902600 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 987/1964] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:17.993423 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc0723-638e-47e7-85bb-eed208b8b13b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:17.995937 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-39fc0723-638e-47e7-85bb-eed208b8b13b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:18.114964 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b7508-36e8-432e-9b27-bd3a8ad543ff service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:18.116311 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-274b7508-36e8-432e-9b27-bd3a8ad543ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:18.134657 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc0723-638e-47e7-85bb-eed208b8b13b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:18.154066 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b7508-36e8-432e-9b27-bd3a8ad543ff service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:18.157070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c0cce-6407-45b0-a250-a80c125c49d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:18.157795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c0cce-6407-45b0-a250-a80c125c49d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.159043 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c0cce-6407-45b0-a250-a80c125c49d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.159043 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c0cce-6407-45b0-a250-a80c125c49d0 admin admin] PATH_INFO: `/v3/projects/758409cca25048d587a5643a972d2e7c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.161986 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c0cce-6407-45b0-a250-a80c125c49d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=758409cca25048d587a5643a972d2e7c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:18.164106 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c0cce-6407-45b0-a250-a80c125c49d0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:18.170170 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e681896-09fc-41e5-8c6b-af47baf4dee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:18.170415 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e681896-09fc-41e5-8c6b-af47baf4dee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.170759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e681896-09fc-41e5-8c6b-af47baf4dee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.171040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e681896-09fc-41e5-8c6b-af47baf4dee3 admin admin] PATH_INFO: `/v3/projects/a6c2b0e41cfa4ec298b81cf5261eba51` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.173005 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e681896-09fc-41e5-8c6b-af47baf4dee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6c2b0e41cfa4ec298b81cf5261eba51)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:18.175153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e681896-09fc-41e5-8c6b-af47baf4dee3 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:18.258633 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:18.277679 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2e681896-09fc-41e5-8c6b-af47baf4dee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6c2b0e41cfa4ec298b81cf5261eba51', 'request_id': 'req-2e681896-09fc-41e5-8c6b-af47baf4dee3'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:18.287829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-878c0cce-6407-45b0-a250-a80c125c49d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '758409cca25048d587a5643a972d2e7c', 'request_id': 'req-878c0cce-6407-45b0-a250-a80c125c49d0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:18.288759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2e681896-09fc-41e5-8c6b-af47baf4dee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6c2b0e41cfa4ec298b81cf5261eba51', 'request_id': 'req-2e681896-09fc-41e5-8c6b-af47baf4dee3'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:18.299091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2e681896-09fc-41e5-8c6b-af47baf4dee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6c2b0e41cfa4ec298b81cf5261eba51', 'request_id': 'req-2e681896-09fc-41e5-8c6b-af47baf4dee3'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:18.300202 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-878c0cce-6407-45b0-a250-a80c125c49d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '758409cca25048d587a5643a972d2e7c', 'request_id': 'req-878c0cce-6407-45b0-a250-a80c125c49d0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:18.317454 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2e681896-09fc-41e5-8c6b-af47baf4dee3 admin admin] The token cache is being invalidate because project a6c2b0e41cfa4ec298b81cf5261eba51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:18.317825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2e681896-09fc-41e5-8c6b-af47baf4dee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:18.319085 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2e681896-09fc-41e5-8c6b-af47baf4dee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:18.321248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 988/1965] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:18 2022] DELETE /identity/v3/projects/a6c2b0e41cfa4ec298b81cf5261eba51 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:18.326583 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-878c0cce-6407-45b0-a250-a80c125c49d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '758409cca25048d587a5643a972d2e7c', 'request_id': 'req-878c0cce-6407-45b0-a250-a80c125c49d0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:18.328634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f11a5-10cb-438e-9fed-d14ad0a0ab27 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.330060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f11a5-10cb-438e-9fed-d14ad0a0ab27 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.330442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f11a5-10cb-438e-9fed-d14ad0a0ab27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.340913 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-878c0cce-6407-45b0-a250-a80c125c49d0 admin admin] The token cache is being invalidate because project 758409cca25048d587a5643a972d2e7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:18.341473 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-878c0cce-6407-45b0-a250-a80c125c49d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:18.342681 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-878c0cce-6407-45b0-a250-a80c125c49d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:18.345279 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 978/1966] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:17 2022] DELETE /identity/v3/projects/758409cca25048d587a5643a972d2e7c => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:18.352532 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61409343-2314-4353-bd24-20ee06836ee6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.353023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61409343-2314-4353-bd24-20ee06836ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.353371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61409343-2314-4353-bd24-20ee06836ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.383944 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-307f11a5-10cb-438e-9fed-d14ad0a0ab27 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:18.386827 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-61409343-2314-4353-bd24-20ee06836ee6 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:18.387914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-307f11a5-10cb-438e-9fed-d14ad0a0ab27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:18.388609 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-61409343-2314-4353-bd24-20ee06836ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:18.458873 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:18.483585 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 979/1967] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:18.485034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2d2d2-cec4-4764-9701-d056c6777236 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.485416 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2d2d2-cec4-4764-9701-d056c6777236 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.486216 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2d2d2-cec4-4764-9701-d056c6777236 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.487186 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 980/1968] 10.222.0.39 () {54 vars in 921 bytes} [Mon Aug 8 02:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:18.491772 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322845b-b163-4855-8c2d-c7b6f0ef57ad None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.492194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322845b-b163-4855-8c2d-c7b6f0ef57ad None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.492563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322845b-b163-4855-8c2d-c7b6f0ef57ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.517126 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d322845b-b163-4855-8c2d-c7b6f0ef57ad None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:18.518588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d322845b-b163-4855-8c2d-c7b6f0ef57ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:18.530513 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 989/1969] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:18.530513 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb09b05-aeb5-4ef6-a436-8d3724988e77 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.530513 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb09b05-aeb5-4ef6-a436-8d3724988e77 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.530513 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb09b05-aeb5-4ef6-a436-8d3724988e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.542200 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 981/1970] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:18.544283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06f1e7-a523-4c1e-8a68-c210f310614c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.545037 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06f1e7-a523-4c1e-8a68-c210f310614c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.545037 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06f1e7-a523-4c1e-8a68-c210f310614c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.556929 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-4eb09b05-aeb5-4ef6-a436-8d3724988e77 None None] MFA Rules not processed for user `c4238566a9c248ffbfa314f47f02670c`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:18.558496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4eb09b05-aeb5-4ef6-a436-8d3724988e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:18.572782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bf06f1e7-a523-4c1e-8a68-c210f310614c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:18.574444 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bf06f1e7-a523-4c1e-8a68-c210f310614c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:18.594448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 982/1971] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:18.594448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06f1e7-a523-4c1e-8a68-c210f310614c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:18.616253 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06f1e7-a523-4c1e-8a68-c210f310614c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:18.630435 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce627a4-e604-4bab-8221-1ed34e47d7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:18.631059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce627a4-e604-4bab-8221-1ed34e47d7fb admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.631336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce627a4-e604-4bab-8221-1ed34e47d7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.631605 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce627a4-e604-4bab-8221-1ed34e47d7fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.633717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce627a4-e604-4bab-8221-1ed34e47d7fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:18.636129 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce627a4-e604-4bab-8221-1ed34e47d7fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:18.637787 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce627a4-e604-4bab-8221-1ed34e47d7fb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:18.670130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 983/1972] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:18.672371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce627a4-e604-4bab-8221-1ed34e47d7fb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:18.686386 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce627a4-e604-4bab-8221-1ed34e47d7fb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:18.703487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 990/1973] 10.222.0.39 () {58 vars in 1018 bytes} [Mon Aug 8 02:26:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:18.704316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb09b05-aeb5-4ef6-a436-8d3724988e77 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:18.707930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783d827-7343-4dfb-971d-109e21e51ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:18.708356 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783d827-7343-4dfb-971d-109e21e51ffd admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.708678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783d827-7343-4dfb-971d-109e21e51ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.708918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783d827-7343-4dfb-971d-109e21e51ffd admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.709578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783d827-7343-4dfb-971d-109e21e51ffd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:18.709922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783d827-7343-4dfb-971d-109e21e51ffd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:18.711689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783d827-7343-4dfb-971d-109e21e51ffd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:18.729142 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 984/1974] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:18.730054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783d827-7343-4dfb-971d-109e21e51ffd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:18.731777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a783d827-7343-4dfb-971d-109e21e51ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:18.738555 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb09b05-aeb5-4ef6-a436-8d3724988e77 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:18.744790 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f1bbc-ffb8-4d99-b4fb-316c3215da6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:18.745604 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f1bbc-ffb8-4d99-b4fb-316c3215da6d admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.746173 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f1bbc-ffb8-4d99-b4fb-316c3215da6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.746807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f1bbc-ffb8-4d99-b4fb-316c3215da6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.747978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f1bbc-ffb8-4d99-b4fb-316c3215da6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:18.748583 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783d827-7343-4dfb-971d-109e21e51ffd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:18.751202 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f1bbc-ffb8-4d99-b4fb-316c3215da6d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:18.754640 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b941291-08da-4a3a-9169-5907a389e9cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:18.755395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b941291-08da-4a3a-9169-5907a389e9cb service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.755680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b941291-08da-4a3a-9169-5907a389e9cb service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.755956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b941291-08da-4a3a-9169-5907a389e9cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.776628 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b941291-08da-4a3a-9169-5907a389e9cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:18.778806 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b941291-08da-4a3a-9169-5907a389e9cb service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:18.791833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 985/1975] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:18.792661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b941291-08da-4a3a-9169-5907a389e9cb service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:18.798519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 991/1976] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:18 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:18.801394 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f1bbc-ffb8-4d99-b4fb-316c3215da6d admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:18.807547 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b941291-08da-4a3a-9169-5907a389e9cb service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:18.860967 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:18.896649 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f1bbc-ffb8-4d99-b4fb-316c3215da6d admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:18.901796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb1e5b-6ca2-421a-8ab5-069184ccb36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:18.903023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb1e5b-6ca2-421a-8ab5-069184ccb36f admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.903357 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb1e5b-6ca2-421a-8ab5-069184ccb36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.903675 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb1e5b-6ca2-421a-8ab5-069184ccb36f admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.904307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb1e5b-6ca2-421a-8ab5-069184ccb36f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:18.906215 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb1e5b-6ca2-421a-8ab5-069184ccb36f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:18.911554 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1468ba-8d06-4c4d-8872-12872a461927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:18.912155 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1468ba-8d06-4c4d-8872-12872a461927 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.912433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1468ba-8d06-4c4d-8872-12872a461927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.912701 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1468ba-8d06-4c4d-8872-12872a461927 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.913384 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1468ba-8d06-4c4d-8872-12872a461927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:18.916025 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1468ba-8d06-4c4d-8872-12872a461927 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:18.928835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 992/1977] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:18 2022] POST /identity/v3/users => generated 287 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:18.935066 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb1e5b-6ca2-421a-8ab5-069184ccb36f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:18.945222 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 986/1978] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:18 2022] POST /identity/v3/projects => generated 377 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:18.945436 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb1e5b-6ca2-421a-8ab5-069184ccb36f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:18.949271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1468ba-8d06-4c4d-8872-12872a461927 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:19.002785 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34617e4-d09a-48b0-8d0e-acc757a4637e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:19.003313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34617e4-d09a-48b0-8d0e-acc757a4637e admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.003594 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34617e4-d09a-48b0-8d0e-acc757a4637e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.003960 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34617e4-d09a-48b0-8d0e-acc757a4637e admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.004606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34617e4-d09a-48b0-8d0e-acc757a4637e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:19.006222 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34617e4-d09a-48b0-8d0e-acc757a4637e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:19.010558 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1468ba-8d06-4c4d-8872-12872a461927 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:19.015173 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d1a3f-f1b1-4649-bc2a-749f00a6e8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:19.016146 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d1a3f-f1b1-4649-bc2a-749f00a6e8bb admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.016420 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d1a3f-f1b1-4649-bc2a-749f00a6e8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.016690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d1a3f-f1b1-4649-bc2a-749f00a6e8bb admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.016809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 993/1979] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:19.017334 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d1a3f-f1b1-4649-bc2a-749f00a6e8bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:19.019277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d1a3f-f1b1-4649-bc2a-749f00a6e8bb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:19.020805 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34617e4-d09a-48b0-8d0e-acc757a4637e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:19.031929 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34617e4-d09a-48b0-8d0e-acc757a4637e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:19.038537 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02c809-865a-416f-8f73-d4a6dc523f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:19.039536 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02c809-865a-416f-8f73-d4a6dc523f15 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.040030 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02c809-865a-416f-8f73-d4a6dc523f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.040974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02c809-865a-416f-8f73-d4a6dc523f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.042030 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02c809-865a-416f-8f73-d4a6dc523f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:19.045058 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02c809-865a-416f-8f73-d4a6dc523f15 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:19.049456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 987/1980] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:18 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:19.053344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d1a3f-f1b1-4649-bc2a-749f00a6e8bb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:19.059609 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 994/1981] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:19.063046 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:19.064744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02c809-865a-416f-8f73-d4a6dc523f15 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:19.067824 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d1a3f-f1b1-4649-bc2a-749f00a6e8bb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:19.074831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76f90e-f8fe-48ee-b7d6-c7be544f953e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:19.075438 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76f90e-f8fe-48ee-b7d6-c7be544f953e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.075715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76f90e-f8fe-48ee-b7d6-c7be544f953e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.075984 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76f90e-f8fe-48ee-b7d6-c7be544f953e admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.076683 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02c809-865a-416f-8f73-d4a6dc523f15 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:19.076787 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76f90e-f8fe-48ee-b7d6-c7be544f953e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:19.078399 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76f90e-f8fe-48ee-b7d6-c7be544f953e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:19.082476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86afc86-afde-4f21-8130-e15b3e6d9b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:19.083022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86afc86-afde-4f21-8130-e15b3e6d9b84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.083415 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86afc86-afde-4f21-8130-e15b3e6d9b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.083832 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86afc86-afde-4f21-8130-e15b3e6d9b84 admin admin] PATH_INFO: `/v3/projects/d3cc8b8bc4964df4a494173af8540c45/users/6916c2afef12490f921d99c5047b4dce/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.091449 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 988/1982] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:19.094856 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76f90e-f8fe-48ee-b7d6-c7be544f953e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:19.102458 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86afc86-afde-4f21-8130-e15b3e6d9b84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3cc8b8bc4964df4a494173af8540c45, user_id=6916c2afef12490f921d99c5047b4dce, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:19.105883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86afc86-afde-4f21-8130-e15b3e6d9b84 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:19.106607 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76f90e-f8fe-48ee-b7d6-c7be544f953e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:19.112450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8612a78-2655-4f19-a73e-d71d23c366f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:19.113099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8612a78-2655-4f19-a73e-d71d23c366f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.113418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8612a78-2655-4f19-a73e-d71d23c366f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.113761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8612a78-2655-4f19-a73e-d71d23c366f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.114596 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8612a78-2655-4f19-a73e-d71d23c366f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:19.116296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8612a78-2655-4f19-a73e-d71d23c366f3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:19.125233 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 995/1983] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:19 2022] PUT /identity/v3/projects/d3cc8b8bc4964df4a494173af8540c45/users/6916c2afef12490f921d99c5047b4dce/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:19.132445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e297a9-0819-4022-8b76-71fed201460f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.132777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e297a9-0819-4022-8b76-71fed201460f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.133051 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e297a9-0819-4022-8b76-71fed201460f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.152937 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 989/1984] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:19.158184 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8612a78-2655-4f19-a73e-d71d23c366f3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:19.158994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-01e297a9-0819-4022-8b76-71fed201460f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:19.160524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-01e297a9-0819-4022-8b76-71fed201460f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:19.172215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8612a78-2655-4f19-a73e-d71d23c366f3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:19.230280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 996/1985] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:19.251483 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e297a9-0819-4022-8b76-71fed201460f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:19.253078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-01e297a9-0819-4022-8b76-71fed201460f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:19.255501 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8800f-0c3b-46c2-b9ca-d974dc2d0280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:19.256165 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8800f-0c3b-46c2-b9ca-d974dc2d0280 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.256444 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8800f-0c3b-46c2-b9ca-d974dc2d0280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.256715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8800f-0c3b-46c2-b9ca-d974dc2d0280 admin admin] PATH_INFO: `/v3/projects/163d60f0461d4b40b0f7e17c015aaef0/users/dfef23227a134421b1e398e206ec0e07/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.273414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8800f-0c3b-46c2-b9ca-d974dc2d0280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=163d60f0461d4b40b0f7e17c015aaef0, user_id=dfef23227a134421b1e398e206ec0e07, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:19.276652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8800f-0c3b-46c2-b9ca-d974dc2d0280 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:19.293350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 990/1986] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:19 2022] PUT /identity/v3/projects/163d60f0461d4b40b0f7e17c015aaef0/users/dfef23227a134421b1e398e206ec0e07/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:19.294554 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:19.303666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d1403-1ae7-462c-8576-2e918004eba2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.303950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d1403-1ae7-462c-8576-2e918004eba2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.304220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d1403-1ae7-462c-8576-2e918004eba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.323701 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e297a9-0819-4022-8b76-71fed201460f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:19.331852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fa1d1403-1ae7-462c-8576-2e918004eba2 None None] MFA Rules not processed for user `dfef23227a134421b1e398e206ec0e07`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:19.333651 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fa1d1403-1ae7-462c-8576-2e918004eba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:19.375021 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cdbf2-65d5-44f9-be0f-f1d16fb78cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:19.375659 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cdbf2-65d5-44f9-be0f-f1d16fb78cfe service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.375940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cdbf2-65d5-44f9-be0f-f1d16fb78cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.376214 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cdbf2-65d5-44f9-be0f-f1d16fb78cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.400984 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 991/1987] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:19.405484 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d1403-1ae7-462c-8576-2e918004eba2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:19.453577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cdbf2-65d5-44f9-be0f-f1d16fb78cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:19.456258 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cdbf2-65d5-44f9-be0f-f1d16fb78cfe service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:19.465794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:19.470602 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 997/1988] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:19.471803 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d1403-1ae7-462c-8576-2e918004eba2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:19.479200 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54417f-6f51-44e5-8592-a623e8a20703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:19.480116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54417f-6f51-44e5-8592-a623e8a20703 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.480426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54417f-6f51-44e5-8592-a623e8a20703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.480760 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54417f-6f51-44e5-8592-a623e8a20703 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.481895 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54417f-6f51-44e5-8592-a623e8a20703 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:19.484321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54417f-6f51-44e5-8592-a623e8a20703 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:19.529846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 992/1989] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:19 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:19.531063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cdbf2-65d5-44f9-be0f-f1d16fb78cfe service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:19.545346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cdbf2-65d5-44f9-be0f-f1d16fb78cfe service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:19.615945 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb932e-5dd0-44f0-8d95-20c3bf7a34e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:19.616588 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb932e-5dd0-44f0-8d95-20c3bf7a34e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.616894 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb932e-5dd0-44f0-8d95-20c3bf7a34e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.617171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb932e-5dd0-44f0-8d95-20c3bf7a34e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.617804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb932e-5dd0-44f0-8d95-20c3bf7a34e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:19.619861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb932e-5dd0-44f0-8d95-20c3bf7a34e2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:19.643943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 998/1990] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:19 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:19.647982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54417f-6f51-44e5-8592-a623e8a20703 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:19.663636 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54417f-6f51-44e5-8592-a623e8a20703 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:19.669065 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01eb56-1250-44a7-9bd9-dc790facdc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:19.669516 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01eb56-1250-44a7-9bd9-dc790facdc77 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.669802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01eb56-1250-44a7-9bd9-dc790facdc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.670074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01eb56-1250-44a7-9bd9-dc790facdc77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.670766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01eb56-1250-44a7-9bd9-dc790facdc77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:19.672383 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01eb56-1250-44a7-9bd9-dc790facdc77 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:19.684312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 993/1991] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:19.688587 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb932e-5dd0-44f0-8d95-20c3bf7a34e2 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:19.702318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb932e-5dd0-44f0-8d95-20c3bf7a34e2 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:19.707734 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dd31f-4c19-49a7-ba09-8f809570d0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:19.708709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dd31f-4c19-49a7-ba09-8f809570d0ad admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.708991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dd31f-4c19-49a7-ba09-8f809570d0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.709258 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dd31f-4c19-49a7-ba09-8f809570d0ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.710169 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dd31f-4c19-49a7-ba09-8f809570d0ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:19.712853 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dd31f-4c19-49a7-ba09-8f809570d0ad admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:19.726156 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 999/1992] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:19.730341 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01eb56-1250-44a7-9bd9-dc790facdc77 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:19.744859 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01eb56-1250-44a7-9bd9-dc790facdc77 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:19.750013 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ee360-6fc9-4767-8e7d-b887c48737e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:19.750709 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ee360-6fc9-4767-8e7d-b887c48737e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.750987 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ee360-6fc9-4767-8e7d-b887c48737e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.751254 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ee360-6fc9-4767-8e7d-b887c48737e5 admin admin] PATH_INFO: `/v3/projects/7f34f8a751d54b9d88b26173204359a9/users/6e9a9a0415604b238bbc17d043d9fca3/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.771700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00ee360-6fc9-4767-8e7d-b887c48737e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f34f8a751d54b9d88b26173204359a9, user_id=6e9a9a0415604b238bbc17d043d9fca3, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:19.775299 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00ee360-6fc9-4767-8e7d-b887c48737e5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:19.791950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 994/1993] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:19 2022] PUT /identity/v3/projects/7f34f8a751d54b9d88b26173204359a9/users/6e9a9a0415604b238bbc17d043d9fca3/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:19.806511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef888e50-a387-4249-8f05-cd6fe7d4457a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.807042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef888e50-a387-4249-8f05-cd6fe7d4457a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.807345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef888e50-a387-4249-8f05-cd6fe7d4457a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.835106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:19.840015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ef888e50-a387-4249-8f05-cd6fe7d4457a None None] MFA Rules not processed for user `6e9a9a0415604b238bbc17d043d9fca3`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:19.842572 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ef888e50-a387-4249-8f05-cd6fe7d4457a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:19.963416 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1000/1994] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:19.968207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ee360-6fc9-4767-8e7d-b887c48737e5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:20.123390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ee360-6fc9-4767-8e7d-b887c48737e5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:20.141980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683f6c9-be0a-4a4b-a412-84cb50b6feca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:20.142300 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683f6c9-be0a-4a4b-a412-84cb50b6feca admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:20.142588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683f6c9-be0a-4a4b-a412-84cb50b6feca admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:20.142865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683f6c9-be0a-4a4b-a412-84cb50b6feca admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:20.146961 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3683f6c9-be0a-4a4b-a412-84cb50b6feca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:20.152021 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3683f6c9-be0a-4a4b-a412-84cb50b6feca admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:20.211664 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 995/1995] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:19 2022] POST /identity/v3/projects => generated 377 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:20.216995 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef888e50-a387-4249-8f05-cd6fe7d4457a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:20.242540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef888e50-a387-4249-8f05-cd6fe7d4457a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:20.418656 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d2153-0a59-4843-8f1e-edc2f1a017d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:20.419578 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d2153-0a59-4843-8f1e-edc2f1a017d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:20.419578 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d2153-0a59-4843-8f1e-edc2f1a017d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:20.419790 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d2153-0a59-4843-8f1e-edc2f1a017d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:20.422681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d2153-0a59-4843-8f1e-edc2f1a017d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:20.424637 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d2153-0a59-4843-8f1e-edc2f1a017d4 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:20.471901 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1001/1996] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:20 2022] POST /identity/v3/users => generated 295 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:20.489359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683f6c9-be0a-4a4b-a412-84cb50b6feca admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:20.507345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683f6c9-be0a-4a4b-a412-84cb50b6feca admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:20.518830 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa0dd5-050f-4c67-b469-c6e03a85c60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:20.519962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa0dd5-050f-4c67-b469-c6e03a85c60d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:20.521332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa0dd5-050f-4c67-b469-c6e03a85c60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:20.521710 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa0dd5-050f-4c67-b469-c6e03a85c60d admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:20.522976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa0dd5-050f-4c67-b469-c6e03a85c60d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:20.534270 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa0dd5-050f-4c67-b469-c6e03a85c60d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:20.544442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 996/1997] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:20.548698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d2153-0a59-4843-8f1e-edc2f1a017d4 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:20.569375 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d2153-0a59-4843-8f1e-edc2f1a017d4 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:20.578222 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38430c-b36a-4c3a-b732-3f8db5c23d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:20.579097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38430c-b36a-4c3a-b732-3f8db5c23d11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:20.579510 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38430c-b36a-4c3a-b732-3f8db5c23d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:20.579878 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38430c-b36a-4c3a-b732-3f8db5c23d11 admin admin] PATH_INFO: `/v3/projects/cec979b8a0e2440bbcda656eedebf060/users/9e9403b3f09e4de1ba75c1dbfda1218d/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:20.600513 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38430c-b36a-4c3a-b732-3f8db5c23d11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec979b8a0e2440bbcda656eedebf060, user_id=9e9403b3f09e4de1ba75c1dbfda1218d, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:20.603203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38430c-b36a-4c3a-b732-3f8db5c23d11 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:20.623355 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1002/1998] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:20 2022] PUT /identity/v3/projects/cec979b8a0e2440bbcda656eedebf060/users/9e9403b3f09e4de1ba75c1dbfda1218d/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:20.627823 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa0dd5-050f-4c67-b469-c6e03a85c60d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:20.640917 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa0dd5-050f-4c67-b469-c6e03a85c60d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:20.712024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749698c-fd32-4add-84d5-cae9a74d56bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:20.712662 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749698c-fd32-4add-84d5-cae9a74d56bf admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:20.712951 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749698c-fd32-4add-84d5-cae9a74d56bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:20.713208 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749698c-fd32-4add-84d5-cae9a74d56bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:20.713864 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749698c-fd32-4add-84d5-cae9a74d56bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:20.715523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749698c-fd32-4add-84d5-cae9a74d56bf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:20.730290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 997/1999] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:20.734008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38430c-b36a-4c3a-b732-3f8db5c23d11 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:20.747109 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38430c-b36a-4c3a-b732-3f8db5c23d11 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:20.752688 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3b08d-b1fb-435b-b597-3958ae622cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:20.753346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3b08d-b1fb-435b-b597-3958ae622cad admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:20.753889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3b08d-b1fb-435b-b597-3958ae622cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:20.754174 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3b08d-b1fb-435b-b597-3958ae622cad admin admin] PATH_INFO: `/v3/projects/cec979b8a0e2440bbcda656eedebf060/users/9e9403b3f09e4de1ba75c1dbfda1218d/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:20.756228 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3b08d-b1fb-435b-b597-3958ae622cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec979b8a0e2440bbcda656eedebf060, user_id=9e9403b3f09e4de1ba75c1dbfda1218d, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:20.759865 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3b08d-b1fb-435b-b597-3958ae622cad admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:20.770998 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1003/2000] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:20 2022] PUT /identity/v3/projects/cec979b8a0e2440bbcda656eedebf060/users/9e9403b3f09e4de1ba75c1dbfda1218d/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:20.781023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d30fac-3e6c-43a6-8477-e062680a9338 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:20.781307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d30fac-3e6c-43a6-8477-e062680a9338 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:20.781583 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d30fac-3e6c-43a6-8477-e062680a9338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:20.820592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-72d30fac-3e6c-43a6-8477-e062680a9338 None None] MFA Rules not processed for user `9e9403b3f09e4de1ba75c1dbfda1218d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:20.823588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-72d30fac-3e6c-43a6-8477-e062680a9338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:20.830630 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:20.862362 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a45181-8685-42d0-91bc-8f0c1434f597 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:20.862762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a45181-8685-42d0-91bc-8f0c1434f597 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:20.863238 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a45181-8685-42d0-91bc-8f0c1434f597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:20.895986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-08a45181-8685-42d0-91bc-8f0c1434f597 None None] MFA Rules not processed for user `6916c2afef12490f921d99c5047b4dce`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:20.897328 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-08a45181-8685-42d0-91bc-8f0c1434f597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:20.925022 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 998/2001] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:20.941416 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d30fac-3e6c-43a6-8477-e062680a9338 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:20.944561 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-72d30fac-3e6c-43a6-8477-e062680a9338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:20.958380 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:20.988226 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1004/2002] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:20.996292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a45181-8685-42d0-91bc-8f0c1434f597 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:20.998227 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-08a45181-8685-42d0-91bc-8f0c1434f597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:21.010729 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d30fac-3e6c-43a6-8477-e062680a9338 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:21.017006 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a45181-8685-42d0-91bc-8f0c1434f597 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:21.023504 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44d5db-4721-4c8a-84be-dc1739c54e02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:21.024261 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44d5db-4721-4c8a-84be-dc1739c54e02 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:21.024678 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44d5db-4721-4c8a-84be-dc1739c54e02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:21.025062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf28e3f-c45f-4f9f-891f-f4c16fd1930d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:21.025384 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44d5db-4721-4c8a-84be-dc1739c54e02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:21.026127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf28e3f-c45f-4f9f-891f-f4c16fd1930d service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:21.026566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf28e3f-c45f-4f9f-891f-f4c16fd1930d service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:21.026683 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf28e3f-c45f-4f9f-891f-f4c16fd1930d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:21.040662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44d5db-4721-4c8a-84be-dc1739c54e02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:21.042837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44d5db-4721-4c8a-84be-dc1739c54e02 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:21.057906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1005/2003] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:21.108410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf28e3f-c45f-4f9f-891f-f4c16fd1930d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:21.111962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf28e3f-c45f-4f9f-891f-f4c16fd1930d service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:21.125057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 999/2004] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:22.515467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80b034-ae56-4b46-aaf5-2da88224af7c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:22.516021 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80b034-ae56-4b46-aaf5-2da88224af7c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:22.516128 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80b034-ae56-4b46-aaf5-2da88224af7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:22.535715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6d80b034-ae56-4b46-aaf5-2da88224af7c None None] MFA Rules not processed for user `6916c2afef12490f921d99c5047b4dce`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:22.536922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6d80b034-ae56-4b46-aaf5-2da88224af7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:22.544591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1006/2005] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:23.817249 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf28e3f-c45f-4f9f-891f-f4c16fd1930d service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:23.826271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf28e3f-c45f-4f9f-891f-f4c16fd1930d service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:23.830137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb035f-b965-49cb-9e31-1a296abf48f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:23.830765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb035f-b965-49cb-9e31-1a296abf48f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:23.831041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb035f-b965-49cb-9e31-1a296abf48f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:23.831313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb035f-b965-49cb-9e31-1a296abf48f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:23.885996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:23.892474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb035f-b965-49cb-9e31-1a296abf48f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:23.894001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb035f-b965-49cb-9e31-1a296abf48f8 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:23.903247 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1000/2006] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:24.685765 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80b034-ae56-4b46-aaf5-2da88224af7c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:24.695763 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80b034-ae56-4b46-aaf5-2da88224af7c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:24.700199 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387a9be-58a1-43ee-ac94-ca692dec6b19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:24.700861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387a9be-58a1-43ee-ac94-ca692dec6b19 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:24.701141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387a9be-58a1-43ee-ac94-ca692dec6b19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:24.701408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387a9be-58a1-43ee-ac94-ca692dec6b19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:24.710748 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387a9be-58a1-43ee-ac94-ca692dec6b19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:24.712200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387a9be-58a1-43ee-ac94-ca692dec6b19 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:24.722561 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1007/2007] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:24 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:28.429656 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb035f-b965-49cb-9e31-1a296abf48f8 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:28.431200 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b3cb035f-b965-49cb-9e31-1a296abf48f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:28.497078 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb035f-b965-49cb-9e31-1a296abf48f8 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:28.501930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f315c-dae7-4872-82af-877ce7be171c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:28.502644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f315c-dae7-4872-82af-877ce7be171c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:28.502955 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f315c-dae7-4872-82af-877ce7be171c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:28.503298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f315c-dae7-4872-82af-877ce7be171c admin admin] PATH_INFO: `/v3/users/1d569f36ac59420788cb2bebf03218a3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:28.504486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f315c-dae7-4872-82af-877ce7be171c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d569f36ac59420788cb2bebf03218a3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:28.506582 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f315c-dae7-4872-82af-877ce7be171c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:28.552502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8f5f315c-dae7-4872-82af-877ce7be171c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d569f36ac59420788cb2bebf03218a3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:28.559557 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8f5f315c-dae7-4872-82af-877ce7be171c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d569f36ac59420788cb2bebf03218a3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:28.566632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8f5f315c-dae7-4872-82af-877ce7be171c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d569f36ac59420788cb2bebf03218a3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:28.567261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8f5f315c-dae7-4872-82af-877ce7be171c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d569f36ac59420788cb2bebf03218a3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:28.567862 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8f5f315c-dae7-4872-82af-877ce7be171c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d569f36ac59420788cb2bebf03218a3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:28.585029 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1001/2008] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:28 2022] DELETE /identity/v3/users/1d569f36ac59420788cb2bebf03218a3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:28.759869 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387a9be-58a1-43ee-ac94-ca692dec6b19 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:28.771806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387a9be-58a1-43ee-ac94-ca692dec6b19 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:28.829432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb504faa-6164-48b9-80c1-f23b9fb7205b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:28.830078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb504faa-6164-48b9-80c1-f23b9fb7205b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:28.830393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb504faa-6164-48b9-80c1-f23b9fb7205b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:28.830674 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb504faa-6164-48b9-80c1-f23b9fb7205b admin admin] PATH_INFO: `/v3/projects/a7a0ae80f10046d1bd9fd4a03e8251dc` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:28.831779 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb504faa-6164-48b9-80c1-f23b9fb7205b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7a0ae80f10046d1bd9fd4a03e8251dc)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:28.833721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb504faa-6164-48b9-80c1-f23b9fb7205b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:28.878224 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-eb504faa-6164-48b9-80c1-f23b9fb7205b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7a0ae80f10046d1bd9fd4a03e8251dc', 'request_id': 'req-eb504faa-6164-48b9-80c1-f23b9fb7205b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:28.883625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-eb504faa-6164-48b9-80c1-f23b9fb7205b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7a0ae80f10046d1bd9fd4a03e8251dc', 'request_id': 'req-eb504faa-6164-48b9-80c1-f23b9fb7205b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:28.891892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-eb504faa-6164-48b9-80c1-f23b9fb7205b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7a0ae80f10046d1bd9fd4a03e8251dc', 'request_id': 'req-eb504faa-6164-48b9-80c1-f23b9fb7205b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:28.902690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-eb504faa-6164-48b9-80c1-f23b9fb7205b admin admin] The token cache is being invalidate because project a7a0ae80f10046d1bd9fd4a03e8251dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:28.902920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-eb504faa-6164-48b9-80c1-f23b9fb7205b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:28.903537 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-eb504faa-6164-48b9-80c1-f23b9fb7205b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:28.904991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1008/2009] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:28 2022] DELETE /identity/v3/projects/a7a0ae80f10046d1bd9fd4a03e8251dc => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:28.909094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f315c-dae7-4872-82af-877ce7be171c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:28.910548 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8f5f315c-dae7-4872-82af-877ce7be171c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:28.923671 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f315c-dae7-4872-82af-877ce7be171c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:28.988737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148a0c2-d65e-4e20-b19f-3eccd612153e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:28.990016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148a0c2-d65e-4e20-b19f-3eccd612153e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:28.990016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148a0c2-d65e-4e20-b19f-3eccd612153e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:28.990515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148a0c2-d65e-4e20-b19f-3eccd612153e admin admin] PATH_INFO: `/v3/users/f7afd87eda944d7fb608d2c853185b97` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:28.991804 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148a0c2-d65e-4e20-b19f-3eccd612153e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7afd87eda944d7fb608d2c853185b97)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:28.993836 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148a0c2-d65e-4e20-b19f-3eccd612153e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:29.008880 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:29.048964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5148a0c2-d65e-4e20-b19f-3eccd612153e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7afd87eda944d7fb608d2c853185b97'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:29.065571 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5148a0c2-d65e-4e20-b19f-3eccd612153e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7afd87eda944d7fb608d2c853185b97'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:29.072898 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5148a0c2-d65e-4e20-b19f-3eccd612153e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7afd87eda944d7fb608d2c853185b97'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:29.074079 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5148a0c2-d65e-4e20-b19f-3eccd612153e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7afd87eda944d7fb608d2c853185b97'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:29.074692 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5148a0c2-d65e-4e20-b19f-3eccd612153e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7afd87eda944d7fb608d2c853185b97'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:29.094441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1002/2010] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:28 2022] DELETE /identity/v3/users/f7afd87eda944d7fb608d2c853185b97 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:29.330573 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb504faa-6164-48b9-80c1-f23b9fb7205b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:29.332029 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-eb504faa-6164-48b9-80c1-f23b9fb7205b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:29.344474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb504faa-6164-48b9-80c1-f23b9fb7205b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:29.392974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:29.412527 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294d3b6-3bc8-4a5a-afe5-0e1a9dc2c801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:29.413182 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294d3b6-3bc8-4a5a-afe5-0e1a9dc2c801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:29.413515 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294d3b6-3bc8-4a5a-afe5-0e1a9dc2c801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:29.413729 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294d3b6-3bc8-4a5a-afe5-0e1a9dc2c801 admin admin] PATH_INFO: `/v3/projects/4cc2c5dc575a42e3baff0668c605caa0` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:29.414896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294d3b6-3bc8-4a5a-afe5-0e1a9dc2c801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cc2c5dc575a42e3baff0668c605caa0)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:29.416812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294d3b6-3bc8-4a5a-afe5-0e1a9dc2c801 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:29.466895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c294d3b6-3bc8-4a5a-afe5-0e1a9dc2c801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cc2c5dc575a42e3baff0668c605caa0', 'request_id': 'req-c294d3b6-3bc8-4a5a-afe5-0e1a9dc2c801'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:29.472413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c294d3b6-3bc8-4a5a-afe5-0e1a9dc2c801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cc2c5dc575a42e3baff0668c605caa0', 'request_id': 'req-c294d3b6-3bc8-4a5a-afe5-0e1a9dc2c801'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:29.480590 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c294d3b6-3bc8-4a5a-afe5-0e1a9dc2c801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cc2c5dc575a42e3baff0668c605caa0', 'request_id': 'req-c294d3b6-3bc8-4a5a-afe5-0e1a9dc2c801'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:29.492204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c294d3b6-3bc8-4a5a-afe5-0e1a9dc2c801 admin admin] The token cache is being invalidate because project 4cc2c5dc575a42e3baff0668c605caa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:29.492525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c294d3b6-3bc8-4a5a-afe5-0e1a9dc2c801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:29.493267 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c294d3b6-3bc8-4a5a-afe5-0e1a9dc2c801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:29.494827 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1009/2011] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:29 2022] DELETE /identity/v3/projects/4cc2c5dc575a42e3baff0668c605caa0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:29.500407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7326031-898a-4da5-8131-3e264cdaba05 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:29.500879 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7326031-898a-4da5-8131-3e264cdaba05 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:29.501002 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7326031-898a-4da5-8131-3e264cdaba05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:29.521452 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a7326031-898a-4da5-8131-3e264cdaba05 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:29.522729 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a7326031-898a-4da5-8131-3e264cdaba05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:29.584428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1003/2012] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:29.594054 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228e535-7c51-4c90-be0b-2fc81b20ef2e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:29.594370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228e535-7c51-4c90-be0b-2fc81b20ef2e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:29.594643 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228e535-7c51-4c90-be0b-2fc81b20ef2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:29.617444 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a228e535-7c51-4c90-be0b-2fc81b20ef2e None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:29.618883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a228e535-7c51-4c90-be0b-2fc81b20ef2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:29.629193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1010/2013] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:29.633438 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7326031-898a-4da5-8131-3e264cdaba05 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:29.643689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7326031-898a-4da5-8131-3e264cdaba05 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:29.648463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd205842-5ba6-43b7-ae35-446be1baa8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:29.649078 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd205842-5ba6-43b7-ae35-446be1baa8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:29.649358 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd205842-5ba6-43b7-ae35-446be1baa8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:29.649630 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd205842-5ba6-43b7-ae35-446be1baa8d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:29.650316 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd205842-5ba6-43b7-ae35-446be1baa8d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:29.650623 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd205842-5ba6-43b7-ae35-446be1baa8d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:29.652740 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd205842-5ba6-43b7-ae35-446be1baa8d0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:29.665321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1004/2014] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:29.670150 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228e535-7c51-4c90-be0b-2fc81b20ef2e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:29.679791 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228e535-7c51-4c90-be0b-2fc81b20ef2e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:29.684605 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d58f5-ec01-45ba-9305-4421e5fcf77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:29.685218 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d58f5-ec01-45ba-9305-4421e5fcf77c admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:29.685500 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d58f5-ec01-45ba-9305-4421e5fcf77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:29.685769 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d58f5-ec01-45ba-9305-4421e5fcf77c admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:29.686433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d58f5-ec01-45ba-9305-4421e5fcf77c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:29.688329 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d58f5-ec01-45ba-9305-4421e5fcf77c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:29.735634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1011/2015] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:29 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:29.736812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd205842-5ba6-43b7-ae35-446be1baa8d0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:29.747363 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd205842-5ba6-43b7-ae35-446be1baa8d0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:29.819163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84dc74-69e1-4e38-a910-60ebf9a24aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:29.819775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84dc74-69e1-4e38-a910-60ebf9a24aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:29.820058 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84dc74-69e1-4e38-a910-60ebf9a24aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:29.820333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84dc74-69e1-4e38-a910-60ebf9a24aa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:29.820955 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84dc74-69e1-4e38-a910-60ebf9a24aa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:29.822932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84dc74-69e1-4e38-a910-60ebf9a24aa9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:29.844743 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1005/2016] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:29 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:29.848178 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d58f5-ec01-45ba-9305-4421e5fcf77c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:29.859043 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d58f5-ec01-45ba-9305-4421e5fcf77c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:29.863615 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df643a76-000c-4539-920a-4e345b1fe1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:29.864230 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df643a76-000c-4539-920a-4e345b1fe1dc admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:29.864512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df643a76-000c-4539-920a-4e345b1fe1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:29.864784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df643a76-000c-4539-920a-4e345b1fe1dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:29.865424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df643a76-000c-4539-920a-4e345b1fe1dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:29.867047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df643a76-000c-4539-920a-4e345b1fe1dc admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:29.876827 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1012/2017] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:29.880430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84dc74-69e1-4e38-a910-60ebf9a24aa9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:29.890373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84dc74-69e1-4e38-a910-60ebf9a24aa9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:29.895059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82036cc4-3ef6-4ae0-8ad9-52866a0fc517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:29.895648 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82036cc4-3ef6-4ae0-8ad9-52866a0fc517 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:29.895929 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82036cc4-3ef6-4ae0-8ad9-52866a0fc517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:29.896203 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82036cc4-3ef6-4ae0-8ad9-52866a0fc517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:29.896839 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82036cc4-3ef6-4ae0-8ad9-52866a0fc517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:29.898469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82036cc4-3ef6-4ae0-8ad9-52866a0fc517 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:29.900910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:29.914642 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1006/2018] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:29.915794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df643a76-000c-4539-920a-4e345b1fe1dc admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:29.926427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df643a76-000c-4539-920a-4e345b1fe1dc admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:29.931016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d7ef9-3658-4f5c-b3b0-9bbbdd4fcc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:29.931589 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d7ef9-3658-4f5c-b3b0-9bbbdd4fcc4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:29.931865 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d7ef9-3658-4f5c-b3b0-9bbbdd4fcc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:29.932135 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d7ef9-3658-4f5c-b3b0-9bbbdd4fcc4e admin admin] PATH_INFO: `/v3/projects/7e8dd43a70594c5d8df8433cfaa446f0/users/49c3eec00bd548859b671b04aca55f5c/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:29.946410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d7ef9-3658-4f5c-b3b0-9bbbdd4fcc4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e8dd43a70594c5d8df8433cfaa446f0, user_id=49c3eec00bd548859b671b04aca55f5c, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:29.949554 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d7ef9-3658-4f5c-b3b0-9bbbdd4fcc4e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:29.962349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1013/2019] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:29 2022] PUT /identity/v3/projects/7e8dd43a70594c5d8df8433cfaa446f0/users/49c3eec00bd548859b671b04aca55f5c/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:29.973309 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0eba4-968c-4539-9de3-aeca15817ead None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:29.973602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0eba4-968c-4539-9de3-aeca15817ead None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:29.973877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0eba4-968c-4539-9de3-aeca15817ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:30.000776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7ba0eba4-968c-4539-9de3-aeca15817ead None None] MFA Rules not processed for user `49c3eec00bd548859b671b04aca55f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:30.001983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7ba0eba4-968c-4539-9de3-aeca15817ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:30.066064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1007/2020] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:30.070615 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d7ef9-3658-4f5c-b3b0-9bbbdd4fcc4e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:30.146086 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d7ef9-3658-4f5c-b3b0-9bbbdd4fcc4e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:30.151396 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdd47e-435c-4366-8d94-c8638c6ca8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:30.153002 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdd47e-435c-4366-8d94-c8638c6ca8e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:30.153600 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdd47e-435c-4366-8d94-c8638c6ca8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:30.153710 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdd47e-435c-4366-8d94-c8638c6ca8e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:30.154484 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fdd47e-435c-4366-8d94-c8638c6ca8e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:30.156887 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fdd47e-435c-4366-8d94-c8638c6ca8e3 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:30.177118 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:30.192142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1014/2021] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:30 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:30.196367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0eba4-968c-4539-9de3-aeca15817ead None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:30.207744 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0eba4-968c-4539-9de3-aeca15817ead None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:30.264628 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ed611-2d32-4881-9d3f-42d1c84f31c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:30.265240 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ed611-2d32-4881-9d3f-42d1c84f31c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:30.265516 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ed611-2d32-4881-9d3f-42d1c84f31c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:30.265900 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ed611-2d32-4881-9d3f-42d1c84f31c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:30.266559 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ed611-2d32-4881-9d3f-42d1c84f31c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:30.268596 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ed611-2d32-4881-9d3f-42d1c84f31c0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:30.290807 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1008/2022] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:30 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:30.295226 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdd47e-435c-4366-8d94-c8638c6ca8e3 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:30.305422 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdd47e-435c-4366-8d94-c8638c6ca8e3 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:30.310567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c946fa-a09d-482c-a31b-b7d76409874a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:30.311158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c946fa-a09d-482c-a31b-b7d76409874a admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:30.311432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c946fa-a09d-482c-a31b-b7d76409874a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:30.311702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c946fa-a09d-482c-a31b-b7d76409874a admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:30.312335 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c946fa-a09d-482c-a31b-b7d76409874a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:30.313889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c946fa-a09d-482c-a31b-b7d76409874a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:30.328442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1015/2023] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:30.329760 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ed611-2d32-4881-9d3f-42d1c84f31c0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:30.339862 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ed611-2d32-4881-9d3f-42d1c84f31c0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:30.344491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6151ee2-7fbe-448c-9c31-8cbea441fc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:30.345149 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6151ee2-7fbe-448c-9c31-8cbea441fc19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:30.345415 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6151ee2-7fbe-448c-9c31-8cbea441fc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:30.345686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6151ee2-7fbe-448c-9c31-8cbea441fc19 admin admin] PATH_INFO: `/v3/projects/231a7594909a4956b4548cdaa37556be/users/5787fadd123a4033baa8d85e852179fa/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:30.360142 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6151ee2-7fbe-448c-9c31-8cbea441fc19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=231a7594909a4956b4548cdaa37556be, user_id=5787fadd123a4033baa8d85e852179fa, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:30.362987 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6151ee2-7fbe-448c-9c31-8cbea441fc19 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:30.379586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1009/2024] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:30 2022] PUT /identity/v3/projects/231a7594909a4956b4548cdaa37556be/users/5787fadd123a4033baa8d85e852179fa/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:30.383628 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c946fa-a09d-482c-a31b-b7d76409874a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:30.394370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c946fa-a09d-482c-a31b-b7d76409874a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:30.460327 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a1d31-860a-468a-b8f5-9f994c7bd25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:30.460933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a1d31-860a-468a-b8f5-9f994c7bd25d admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:30.461210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a1d31-860a-468a-b8f5-9f994c7bd25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:30.461479 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a1d31-860a-468a-b8f5-9f994c7bd25d admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:30.462116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a1d31-860a-468a-b8f5-9f994c7bd25d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:30.463801 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a1d31-860a-468a-b8f5-9f994c7bd25d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:30.474221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1016/2025] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:30.478127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6151ee2-7fbe-448c-9c31-8cbea441fc19 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:30.488458 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6151ee2-7fbe-448c-9c31-8cbea441fc19 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:30.493076 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f3f2e-5dcb-44a0-bdf1-2b7361ecfe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:30.493721 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f3f2e-5dcb-44a0-bdf1-2b7361ecfe2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:30.494021 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f3f2e-5dcb-44a0-bdf1-2b7361ecfe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:30.494321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f3f2e-5dcb-44a0-bdf1-2b7361ecfe2d admin admin] PATH_INFO: `/v3/projects/231a7594909a4956b4548cdaa37556be/users/5787fadd123a4033baa8d85e852179fa/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:30.496753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f3f2e-5dcb-44a0-bdf1-2b7361ecfe2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=231a7594909a4956b4548cdaa37556be, user_id=5787fadd123a4033baa8d85e852179fa, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:30.499422 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f3f2e-5dcb-44a0-bdf1-2b7361ecfe2d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:30.509704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1010/2026] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:30 2022] PUT /identity/v3/projects/231a7594909a4956b4548cdaa37556be/users/5787fadd123a4033baa8d85e852179fa/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:30.520693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982a14d-cf23-4445-be41-0565080b5f2e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:30.520978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982a14d-cf23-4445-be41-0565080b5f2e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:30.521282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982a14d-cf23-4445-be41-0565080b5f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:30.549230 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a982a14d-cf23-4445-be41-0565080b5f2e None None] MFA Rules not processed for user `5787fadd123a4033baa8d85e852179fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:30.550488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a982a14d-cf23-4445-be41-0565080b5f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:30.628657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1017/2027] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:30.640436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f3f2e-5dcb-44a0-bdf1-2b7361ecfe2d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:30.642128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-215f3f2e-5dcb-44a0-bdf1-2b7361ecfe2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:30.685897 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:30.702512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f3f2e-5dcb-44a0-bdf1-2b7361ecfe2d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:30.706210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7ca41-7101-40d0-b1da-f5f02f16849b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:30.706843 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7ca41-7101-40d0-b1da-f5f02f16849b service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:30.707153 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7ca41-7101-40d0-b1da-f5f02f16849b service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:30.707425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7ca41-7101-40d0-b1da-f5f02f16849b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:30.766531 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7ca41-7101-40d0-b1da-f5f02f16849b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:30.767974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7ca41-7101-40d0-b1da-f5f02f16849b service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:30.776979 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1011/2028] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:32.906226 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982a14d-cf23-4445-be41-0565080b5f2e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:32.907709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a982a14d-cf23-4445-be41-0565080b5f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:32.972860 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982a14d-cf23-4445-be41-0565080b5f2e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:32.977422 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780e3ed-d835-498c-9f61-75df796b6a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:32.978042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780e3ed-d835-498c-9f61-75df796b6a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:32.978380 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780e3ed-d835-498c-9f61-75df796b6a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:32.978652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780e3ed-d835-498c-9f61-75df796b6a83 admin admin] PATH_INFO: `/v3/users/dfef23227a134421b1e398e206ec0e07` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:32.980255 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780e3ed-d835-498c-9f61-75df796b6a83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfef23227a134421b1e398e206ec0e07)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:32.982155 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780e3ed-d835-498c-9f61-75df796b6a83 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:32.986744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:33.028484 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7780e3ed-d835-498c-9f61-75df796b6a83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfef23227a134421b1e398e206ec0e07'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.035237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7780e3ed-d835-498c-9f61-75df796b6a83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfef23227a134421b1e398e206ec0e07'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.042739 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7780e3ed-d835-498c-9f61-75df796b6a83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfef23227a134421b1e398e206ec0e07'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.043365 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7780e3ed-d835-498c-9f61-75df796b6a83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfef23227a134421b1e398e206ec0e07'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.043979 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7780e3ed-d835-498c-9f61-75df796b6a83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfef23227a134421b1e398e206ec0e07'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.061654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1018/2029] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:32 2022] DELETE /identity/v3/users/dfef23227a134421b1e398e206ec0e07 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:33.066537 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4479a57-d616-4ae1-aaca-6bbdc8ddf94c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:33.067041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4479a57-d616-4ae1-aaca-6bbdc8ddf94c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:33.067133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4479a57-d616-4ae1-aaca-6bbdc8ddf94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:33.085398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f4479a57-d616-4ae1-aaca-6bbdc8ddf94c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:33.086615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f4479a57-d616-4ae1-aaca-6bbdc8ddf94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:33.143244 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1012/2030] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:33.153514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780e3ed-d835-498c-9f61-75df796b6a83 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:33.155934 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7780e3ed-d835-498c-9f61-75df796b6a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:33.223014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780e3ed-d835-498c-9f61-75df796b6a83 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:33.229320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c97072-c570-4252-b253-020c7ad34e4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:33.230563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c97072-c570-4252-b253-020c7ad34e4c service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:33.230760 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c97072-c570-4252-b253-020c7ad34e4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:33.231031 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c97072-c570-4252-b253-020c7ad34e4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:33.244239 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c97072-c570-4252-b253-020c7ad34e4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:33.246343 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c97072-c570-4252-b253-020c7ad34e4c service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:33.257186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1019/2031] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:33.632673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4479a57-d616-4ae1-aaca-6bbdc8ddf94c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:33.633780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f4479a57-d616-4ae1-aaca-6bbdc8ddf94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:33.655079 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4479a57-d616-4ae1-aaca-6bbdc8ddf94c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:33.684539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4731fab-727e-44b2-bf0d-247c5182b430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:33.685186 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4731fab-727e-44b2-bf0d-247c5182b430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:33.685465 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4731fab-727e-44b2-bf0d-247c5182b430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:33.685736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4731fab-727e-44b2-bf0d-247c5182b430 admin admin] PATH_INFO: `/v3/projects/163d60f0461d4b40b0f7e17c015aaef0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:33.688273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4731fab-727e-44b2-bf0d-247c5182b430 admin admin] RBAC: Authorizing `identity:delete_project(project_id=163d60f0461d4b40b0f7e17c015aaef0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:33.689858 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4731fab-727e-44b2-bf0d-247c5182b430 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:33.767866 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4731fab-727e-44b2-bf0d-247c5182b430 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '163d60f0461d4b40b0f7e17c015aaef0', 'request_id': 'req-c4731fab-727e-44b2-bf0d-247c5182b430'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.777492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4731fab-727e-44b2-bf0d-247c5182b430 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '163d60f0461d4b40b0f7e17c015aaef0', 'request_id': 'req-c4731fab-727e-44b2-bf0d-247c5182b430'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.790981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4731fab-727e-44b2-bf0d-247c5182b430 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '163d60f0461d4b40b0f7e17c015aaef0', 'request_id': 'req-c4731fab-727e-44b2-bf0d-247c5182b430'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.806327 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4731fab-727e-44b2-bf0d-247c5182b430 admin admin] The token cache is being invalidate because project 163d60f0461d4b40b0f7e17c015aaef0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:33.806622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4731fab-727e-44b2-bf0d-247c5182b430 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.807938 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4731fab-727e-44b2-bf0d-247c5182b430 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.810327 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1013/2032] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:33 2022] DELETE /identity/v3/projects/163d60f0461d4b40b0f7e17c015aaef0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:33.814437 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c97072-c570-4252-b253-020c7ad34e4c service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:33.825658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c97072-c570-4252-b253-020c7ad34e4c service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:33.889909 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b94e0-243f-42c6-a831-b5b40ddc607a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:33.890980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b94e0-243f-42c6-a831-b5b40ddc607a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:33.891645 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b94e0-243f-42c6-a831-b5b40ddc607a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:33.891831 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b94e0-243f-42c6-a831-b5b40ddc607a admin admin] PATH_INFO: `/v3/users/6e9a9a0415604b238bbc17d043d9fca3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:33.893507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b94e0-243f-42c6-a831-b5b40ddc607a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e9a9a0415604b238bbc17d043d9fca3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:33.895903 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b94e0-243f-42c6-a831-b5b40ddc607a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:33.926062 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:33.947902 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d92b94e0-243f-42c6-a831-b5b40ddc607a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e9a9a0415604b238bbc17d043d9fca3'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.962011 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d92b94e0-243f-42c6-a831-b5b40ddc607a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e9a9a0415604b238bbc17d043d9fca3'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.973346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d92b94e0-243f-42c6-a831-b5b40ddc607a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e9a9a0415604b238bbc17d043d9fca3'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.974759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d92b94e0-243f-42c6-a831-b5b40ddc607a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e9a9a0415604b238bbc17d043d9fca3'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.975583 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d92b94e0-243f-42c6-a831-b5b40ddc607a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e9a9a0415604b238bbc17d043d9fca3'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.994809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1020/2033] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:33 2022] DELETE /identity/v3/users/6e9a9a0415604b238bbc17d043d9fca3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:34.284014 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4731fab-727e-44b2-bf0d-247c5182b430 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:34.285500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c4731fab-727e-44b2-bf0d-247c5182b430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:34.298688 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4731fab-727e-44b2-bf0d-247c5182b430 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:34.312423 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:34.358566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94753518-63ad-47f7-bab7-38959d30e69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:34.359192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94753518-63ad-47f7-bab7-38959d30e69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:34.359469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94753518-63ad-47f7-bab7-38959d30e69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:34.359738 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94753518-63ad-47f7-bab7-38959d30e69d admin admin] PATH_INFO: `/v3/projects/7f34f8a751d54b9d88b26173204359a9` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:34.360845 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94753518-63ad-47f7-bab7-38959d30e69d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f34f8a751d54b9d88b26173204359a9)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:34.362861 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94753518-63ad-47f7-bab7-38959d30e69d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:34.408861 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-94753518-63ad-47f7-bab7-38959d30e69d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f34f8a751d54b9d88b26173204359a9', 'request_id': 'req-94753518-63ad-47f7-bab7-38959d30e69d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:34.416178 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-94753518-63ad-47f7-bab7-38959d30e69d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f34f8a751d54b9d88b26173204359a9', 'request_id': 'req-94753518-63ad-47f7-bab7-38959d30e69d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:34.423215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-94753518-63ad-47f7-bab7-38959d30e69d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f34f8a751d54b9d88b26173204359a9', 'request_id': 'req-94753518-63ad-47f7-bab7-38959d30e69d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:34.435000 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-94753518-63ad-47f7-bab7-38959d30e69d admin admin] The token cache is being invalidate because project 7f34f8a751d54b9d88b26173204359a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:34.435309 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-94753518-63ad-47f7-bab7-38959d30e69d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:34.435932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-94753518-63ad-47f7-bab7-38959d30e69d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:34.437400 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1014/2034] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:34 2022] DELETE /identity/v3/projects/7f34f8a751d54b9d88b26173204359a9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:34.441658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b94e0-243f-42c6-a831-b5b40ddc607a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:34.443123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d92b94e0-243f-42c6-a831-b5b40ddc607a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:34.456684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b94e0-243f-42c6-a831-b5b40ddc607a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:34.514927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b44965-e89a-4e56-82f2-5d06bc234fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:34.515490 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b44965-e89a-4e56-82f2-5d06bc234fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:34.515787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b44965-e89a-4e56-82f2-5d06bc234fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:34.516038 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b44965-e89a-4e56-82f2-5d06bc234fb0 admin admin] PATH_INFO: `/v3/users/9e9403b3f09e4de1ba75c1dbfda1218d` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:34.517145 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b44965-e89a-4e56-82f2-5d06bc234fb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e9403b3f09e4de1ba75c1dbfda1218d)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:34.519098 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b44965-e89a-4e56-82f2-5d06bc234fb0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:34.569490 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-18b44965-e89a-4e56-82f2-5d06bc234fb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e9403b3f09e4de1ba75c1dbfda1218d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:34.576724 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-18b44965-e89a-4e56-82f2-5d06bc234fb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e9403b3f09e4de1ba75c1dbfda1218d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:34.584920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-18b44965-e89a-4e56-82f2-5d06bc234fb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e9403b3f09e4de1ba75c1dbfda1218d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:34.585568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-18b44965-e89a-4e56-82f2-5d06bc234fb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e9403b3f09e4de1ba75c1dbfda1218d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:34.586167 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-18b44965-e89a-4e56-82f2-5d06bc234fb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e9403b3f09e4de1ba75c1dbfda1218d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:34.610065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1021/2035] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:34 2022] DELETE /identity/v3/users/9e9403b3f09e4de1ba75c1dbfda1218d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:34.888261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94753518-63ad-47f7-bab7-38959d30e69d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:34.890277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-94753518-63ad-47f7-bab7-38959d30e69d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:34.922044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94753518-63ad-47f7-bab7-38959d30e69d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:34.987446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ae060-e246-4a0c-9051-37fb3bc58513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:34.988281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ae060-e246-4a0c-9051-37fb3bc58513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:34.988649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ae060-e246-4a0c-9051-37fb3bc58513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:34.988990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ae060-e246-4a0c-9051-37fb3bc58513 admin admin] PATH_INFO: `/v3/projects/cec979b8a0e2440bbcda656eedebf060` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:34.990326 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ae060-e246-4a0c-9051-37fb3bc58513 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cec979b8a0e2440bbcda656eedebf060)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:34.992336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ae060-e246-4a0c-9051-37fb3bc58513 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:35.051703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c55ae060-e246-4a0c-9051-37fb3bc58513 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cec979b8a0e2440bbcda656eedebf060', 'request_id': 'req-c55ae060-e246-4a0c-9051-37fb3bc58513'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:35.057733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c55ae060-e246-4a0c-9051-37fb3bc58513 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cec979b8a0e2440bbcda656eedebf060', 'request_id': 'req-c55ae060-e246-4a0c-9051-37fb3bc58513'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:35.063053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:35.068576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c55ae060-e246-4a0c-9051-37fb3bc58513 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cec979b8a0e2440bbcda656eedebf060', 'request_id': 'req-c55ae060-e246-4a0c-9051-37fb3bc58513'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:35.084526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c55ae060-e246-4a0c-9051-37fb3bc58513 admin admin] The token cache is being invalidate because project cec979b8a0e2440bbcda656eedebf060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:35.084970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c55ae060-e246-4a0c-9051-37fb3bc58513 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:35.086572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c55ae060-e246-4a0c-9051-37fb3bc58513 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:35.092025 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1015/2036] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:34 2022] DELETE /identity/v3/projects/cec979b8a0e2440bbcda656eedebf060 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:35.096742 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d7eab-1c1d-44cd-9007-0492bbb7aa37 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:35.097240 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d7eab-1c1d-44cd-9007-0492bbb7aa37 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:35.097331 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d7eab-1c1d-44cd-9007-0492bbb7aa37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:35.120374 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-080d7eab-1c1d-44cd-9007-0492bbb7aa37 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:35.121770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-080d7eab-1c1d-44cd-9007-0492bbb7aa37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:35.143237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:35.200522 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1022/2037] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:35.209209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841ab82-3c69-4020-9a1e-bdb3b1247b23 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:35.209489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841ab82-3c69-4020-9a1e-bdb3b1247b23 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:35.209766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841ab82-3c69-4020-9a1e-bdb3b1247b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:35.237588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0841ab82-3c69-4020-9a1e-bdb3b1247b23 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:35.238761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0841ab82-3c69-4020-9a1e-bdb3b1247b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:35.246733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1016/2038] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:35.250944 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d7eab-1c1d-44cd-9007-0492bbb7aa37 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:35.260682 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d7eab-1c1d-44cd-9007-0492bbb7aa37 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:35.265906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75371017-c9af-4cb6-adf6-2ec2c5de0927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:35.266538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75371017-c9af-4cb6-adf6-2ec2c5de0927 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:35.266813 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75371017-c9af-4cb6-adf6-2ec2c5de0927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:35.267077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75371017-c9af-4cb6-adf6-2ec2c5de0927 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:35.267770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75371017-c9af-4cb6-adf6-2ec2c5de0927 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:35.268067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75371017-c9af-4cb6-adf6-2ec2c5de0927 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:35.269631 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75371017-c9af-4cb6-adf6-2ec2c5de0927 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:35.283260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1023/2039] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:35.287523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841ab82-3c69-4020-9a1e-bdb3b1247b23 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:35.296780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841ab82-3c69-4020-9a1e-bdb3b1247b23 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:35.301442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d81440-26c2-4a2a-9376-76054f85f38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:35.302030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d81440-26c2-4a2a-9376-76054f85f38d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:35.302329 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d81440-26c2-4a2a-9376-76054f85f38d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:35.302599 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d81440-26c2-4a2a-9376-76054f85f38d admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:35.303213 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d81440-26c2-4a2a-9376-76054f85f38d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:35.305109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d81440-26c2-4a2a-9376-76054f85f38d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:35.335335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1017/2040] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:35 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:35.340495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75371017-c9af-4cb6-adf6-2ec2c5de0927 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:35.350351 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75371017-c9af-4cb6-adf6-2ec2c5de0927 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:35.406308 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d31f9-9cf4-4452-92e3-ac0e6d0d7fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:35.406919 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d31f9-9cf4-4452-92e3-ac0e6d0d7fcd admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:35.407194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d31f9-9cf4-4452-92e3-ac0e6d0d7fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:35.407460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d31f9-9cf4-4452-92e3-ac0e6d0d7fcd admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:35.408186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d31f9-9cf4-4452-92e3-ac0e6d0d7fcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:35.410217 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d31f9-9cf4-4452-92e3-ac0e6d0d7fcd admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:35.433123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1024/2041] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:35 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:35.436475 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d81440-26c2-4a2a-9376-76054f85f38d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:35.447357 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d81440-26c2-4a2a-9376-76054f85f38d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:35.452426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e733ee8-f20e-4fe0-b468-79cd05278e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:35.453011 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e733ee8-f20e-4fe0-b468-79cd05278e76 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:35.453285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e733ee8-f20e-4fe0-b468-79cd05278e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:35.453550 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e733ee8-f20e-4fe0-b468-79cd05278e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:35.454182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e733ee8-f20e-4fe0-b468-79cd05278e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:35.455787 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e733ee8-f20e-4fe0-b468-79cd05278e76 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:35.466310 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1018/2042] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:35.470182 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d31f9-9cf4-4452-92e3-ac0e6d0d7fcd admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:35.480130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d31f9-9cf4-4452-92e3-ac0e6d0d7fcd admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:35.484656 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e02753-00de-46d3-b970-86483d921e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:35.485240 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e02753-00de-46d3-b970-86483d921e4d admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:35.485516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e02753-00de-46d3-b970-86483d921e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:35.485786 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e02753-00de-46d3-b970-86483d921e4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:35.486456 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e02753-00de-46d3-b970-86483d921e4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:35.488067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e02753-00de-46d3-b970-86483d921e4d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:35.498388 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1025/2043] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:35.502172 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e733ee8-f20e-4fe0-b468-79cd05278e76 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:35.512685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e733ee8-f20e-4fe0-b468-79cd05278e76 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:35.517205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35f8d1-59cc-4901-9d34-1bb302ec3643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:35.517834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35f8d1-59cc-4901-9d34-1bb302ec3643 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:35.518108 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35f8d1-59cc-4901-9d34-1bb302ec3643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:35.518411 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35f8d1-59cc-4901-9d34-1bb302ec3643 admin admin] PATH_INFO: `/v3/projects/c4a56db115f64261ae4b472567a2265e/users/2b69dd6e1f7c4d93af3a27f310a7704c/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:35.533367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35f8d1-59cc-4901-9d34-1bb302ec3643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4a56db115f64261ae4b472567a2265e, user_id=2b69dd6e1f7c4d93af3a27f310a7704c, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:35.536063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35f8d1-59cc-4901-9d34-1bb302ec3643 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:35.548674 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1019/2044] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:35 2022] PUT /identity/v3/projects/c4a56db115f64261ae4b472567a2265e/users/2b69dd6e1f7c4d93af3a27f310a7704c/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:35.559376 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225595ac-14b0-419e-be55-7e7787ea2c06 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:35.559688 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225595ac-14b0-419e-be55-7e7787ea2c06 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:35.559961 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225595ac-14b0-419e-be55-7e7787ea2c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:35.587773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-225595ac-14b0-419e-be55-7e7787ea2c06 None None] MFA Rules not processed for user `2b69dd6e1f7c4d93af3a27f310a7704c`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:35.589146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-225595ac-14b0-419e-be55-7e7787ea2c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:35.652084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1026/2045] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:35.662453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35f8d1-59cc-4901-9d34-1bb302ec3643 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:35.664187 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8e35f8d1-59cc-4901-9d34-1bb302ec3643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:35.722660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35f8d1-59cc-4901-9d34-1bb302ec3643 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:35.726576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6a9c6-6b7f-4aac-b479-683ba7464757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:35.727181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6a9c6-6b7f-4aac-b479-683ba7464757 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:35.727458 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6a9c6-6b7f-4aac-b479-683ba7464757 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:35.727726 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6a9c6-6b7f-4aac-b479-683ba7464757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:35.736852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6a9c6-6b7f-4aac-b479-683ba7464757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:35.738455 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6a9c6-6b7f-4aac-b479-683ba7464757 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:35.747871 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1020/2046] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:39.217935 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefc827-6fd9-4d76-9db8-899c7acb376c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:39.218429 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefc827-6fd9-4d76-9db8-899c7acb376c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:39.218562 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefc827-6fd9-4d76-9db8-899c7acb376c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:39.239740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-aaefc827-6fd9-4d76-9db8-899c7acb376c None None] MFA Rules not processed for user `2b69dd6e1f7c4d93af3a27f310a7704c`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:39.240765 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-aaefc827-6fd9-4d76-9db8-899c7acb376c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:39.248676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1027/2047] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:40.514292 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6a9c6-6b7f-4aac-b479-683ba7464757 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:40.515865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5cb6a9c6-6b7f-4aac-b479-683ba7464757 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:40.528759 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6a9c6-6b7f-4aac-b479-683ba7464757 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:40.534004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ef022-eb91-4644-aecb-0208ee09b95e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:40.534784 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ef022-eb91-4644-aecb-0208ee09b95e service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:40.535069 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ef022-eb91-4644-aecb-0208ee09b95e service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:40.535345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ef022-eb91-4644-aecb-0208ee09b95e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:40.537115 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f90ef022-eb91-4644-aecb-0208ee09b95e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:40.624625 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ef022-eb91-4644-aecb-0208ee09b95e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:40.626319 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ef022-eb91-4644-aecb-0208ee09b95e service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:40.642773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1021/2048] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:45.687717 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefc827-6fd9-4d76-9db8-899c7acb376c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:45.689588 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-aaefc827-6fd9-4d76-9db8-899c7acb376c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:45.712264 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:45.820124 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefc827-6fd9-4d76-9db8-899c7acb376c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:45.833685 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374dbd7-76be-4685-bb2d-5bb372b40123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:45.836198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374dbd7-76be-4685-bb2d-5bb372b40123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:45.837906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374dbd7-76be-4685-bb2d-5bb372b40123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:45.838365 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374dbd7-76be-4685-bb2d-5bb372b40123 admin admin] PATH_INFO: `/v3/users/49c3eec00bd548859b671b04aca55f5c` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:45.841659 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374dbd7-76be-4685-bb2d-5bb372b40123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49c3eec00bd548859b671b04aca55f5c)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:45.847317 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374dbd7-76be-4685-bb2d-5bb372b40123 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:45.964861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d374dbd7-76be-4685-bb2d-5bb372b40123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49c3eec00bd548859b671b04aca55f5c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:45.978147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d374dbd7-76be-4685-bb2d-5bb372b40123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49c3eec00bd548859b671b04aca55f5c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:46.000509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d374dbd7-76be-4685-bb2d-5bb372b40123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49c3eec00bd548859b671b04aca55f5c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:46.001365 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d374dbd7-76be-4685-bb2d-5bb372b40123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49c3eec00bd548859b671b04aca55f5c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:46.004401 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d374dbd7-76be-4685-bb2d-5bb372b40123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49c3eec00bd548859b671b04aca55f5c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:46.066370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1028/2049] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:45 2022] DELETE /identity/v3/users/49c3eec00bd548859b671b04aca55f5c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:46.068491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e08c3-be9e-4460-b36f-9035d4c5b4f2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:46.069017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e08c3-be9e-4460-b36f-9035d4c5b4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:46.069278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e08c3-be9e-4460-b36f-9035d4c5b4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:46.116429 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0e0e08c3-be9e-4460-b36f-9035d4c5b4f2 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:46.120028 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0e0e08c3-be9e-4460-b36f-9035d4c5b4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:46.129699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:46.211774 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1022/2050] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:46.239751 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374dbd7-76be-4685-bb2d-5bb372b40123 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:46.241407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d374dbd7-76be-4685-bb2d-5bb372b40123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:46.359536 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374dbd7-76be-4685-bb2d-5bb372b40123 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:46.364106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74096d9-e8cb-43ac-9eca-afbdacaf3516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:46.364990 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74096d9-e8cb-43ac-9eca-afbdacaf3516 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:46.365447 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74096d9-e8cb-43ac-9eca-afbdacaf3516 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:46.365890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74096d9-e8cb-43ac-9eca-afbdacaf3516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:46.395215 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74096d9-e8cb-43ac-9eca-afbdacaf3516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:46.396656 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74096d9-e8cb-43ac-9eca-afbdacaf3516 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:46.422413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1029/2051] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:47.541974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e08c3-be9e-4460-b36f-9035d4c5b4f2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:47.543455 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0e0e08c3-be9e-4460-b36f-9035d4c5b4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:47.555990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e08c3-be9e-4460-b36f-9035d4c5b4f2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:47.561127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515458a-b940-431c-947e-00767688d36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:47.561764 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515458a-b940-431c-947e-00767688d36f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:47.562043 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515458a-b940-431c-947e-00767688d36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:47.562841 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515458a-b940-431c-947e-00767688d36f admin admin] PATH_INFO: `/v3/projects/7e8dd43a70594c5d8df8433cfaa446f0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:47.563989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a515458a-b940-431c-947e-00767688d36f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e8dd43a70594c5d8df8433cfaa446f0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:47.566173 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a515458a-b940-431c-947e-00767688d36f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:47.624247 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a515458a-b940-431c-947e-00767688d36f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e8dd43a70594c5d8df8433cfaa446f0', 'request_id': 'req-a515458a-b940-431c-947e-00767688d36f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:47.629856 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a515458a-b940-431c-947e-00767688d36f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e8dd43a70594c5d8df8433cfaa446f0', 'request_id': 'req-a515458a-b940-431c-947e-00767688d36f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:47.643471 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a515458a-b940-431c-947e-00767688d36f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e8dd43a70594c5d8df8433cfaa446f0', 'request_id': 'req-a515458a-b940-431c-947e-00767688d36f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:47.656353 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a515458a-b940-431c-947e-00767688d36f admin admin] The token cache is being invalidate because project 7e8dd43a70594c5d8df8433cfaa446f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:47.656756 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a515458a-b940-431c-947e-00767688d36f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:47.657394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a515458a-b940-431c-947e-00767688d36f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:47.658927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1023/2052] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:47 2022] DELETE /identity/v3/projects/7e8dd43a70594c5d8df8433cfaa446f0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:47.663005 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74096d9-e8cb-43ac-9eca-afbdacaf3516 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:47.673747 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74096d9-e8cb-43ac-9eca-afbdacaf3516 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:47.715853 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:47.746603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad3670-2713-4132-a2a9-ed93d9a4b488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:47.747608 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad3670-2713-4132-a2a9-ed93d9a4b488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:47.748073 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad3670-2713-4132-a2a9-ed93d9a4b488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:47.748528 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad3670-2713-4132-a2a9-ed93d9a4b488 admin admin] PATH_INFO: `/v3/users/5787fadd123a4033baa8d85e852179fa` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:47.749883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad3670-2713-4132-a2a9-ed93d9a4b488 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5787fadd123a4033baa8d85e852179fa)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:47.752087 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad3670-2713-4132-a2a9-ed93d9a4b488 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:47.827821 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d3ad3670-2713-4132-a2a9-ed93d9a4b488 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5787fadd123a4033baa8d85e852179fa'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:47.835559 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d3ad3670-2713-4132-a2a9-ed93d9a4b488 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5787fadd123a4033baa8d85e852179fa'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:47.844053 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d3ad3670-2713-4132-a2a9-ed93d9a4b488 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5787fadd123a4033baa8d85e852179fa'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:47.844750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d3ad3670-2713-4132-a2a9-ed93d9a4b488 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5787fadd123a4033baa8d85e852179fa'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:47.845491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d3ad3670-2713-4132-a2a9-ed93d9a4b488 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5787fadd123a4033baa8d85e852179fa'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:47.865086 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1030/2053] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:47 2022] DELETE /identity/v3/users/5787fadd123a4033baa8d85e852179fa => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:48.143460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515458a-b940-431c-947e-00767688d36f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:48.145347 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a515458a-b940-431c-947e-00767688d36f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:48.181349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515458a-b940-431c-947e-00767688d36f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:48.248099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd191a28-565e-4da5-93a6-1cffc12731e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:48.248773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd191a28-565e-4da5-93a6-1cffc12731e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:48.249055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd191a28-565e-4da5-93a6-1cffc12731e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:48.249327 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd191a28-565e-4da5-93a6-1cffc12731e6 admin admin] PATH_INFO: `/v3/projects/231a7594909a4956b4548cdaa37556be` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:48.250713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd191a28-565e-4da5-93a6-1cffc12731e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=231a7594909a4956b4548cdaa37556be)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:48.253247 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd191a28-565e-4da5-93a6-1cffc12731e6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:48.292171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:48.306791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cd191a28-565e-4da5-93a6-1cffc12731e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '231a7594909a4956b4548cdaa37556be', 'request_id': 'req-cd191a28-565e-4da5-93a6-1cffc12731e6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:48.313196 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cd191a28-565e-4da5-93a6-1cffc12731e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '231a7594909a4956b4548cdaa37556be', 'request_id': 'req-cd191a28-565e-4da5-93a6-1cffc12731e6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:48.322634 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cd191a28-565e-4da5-93a6-1cffc12731e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '231a7594909a4956b4548cdaa37556be', 'request_id': 'req-cd191a28-565e-4da5-93a6-1cffc12731e6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:48.334634 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cd191a28-565e-4da5-93a6-1cffc12731e6 admin admin] The token cache is being invalidate because project 231a7594909a4956b4548cdaa37556be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:48.334927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cd191a28-565e-4da5-93a6-1cffc12731e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:48.335566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cd191a28-565e-4da5-93a6-1cffc12731e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:48.337221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1024/2054] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:48 2022] DELETE /identity/v3/projects/231a7594909a4956b4548cdaa37556be => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:48.345415 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b49a5b-8bb6-44db-b8fa-9cc7159c338c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:48.345875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b49a5b-8bb6-44db-b8fa-9cc7159c338c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:48.346349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b49a5b-8bb6-44db-b8fa-9cc7159c338c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:48.368558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-95b49a5b-8bb6-44db-b8fa-9cc7159c338c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:48.370464 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-95b49a5b-8bb6-44db-b8fa-9cc7159c338c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:48.430487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1031/2055] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:48.439656 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f9594-305c-4bd4-9959-2fbc3132e11b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:48.440474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f9594-305c-4bd4-9959-2fbc3132e11b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:48.440474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f9594-305c-4bd4-9959-2fbc3132e11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:48.460442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6f5f9594-305c-4bd4-9959-2fbc3132e11b None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:48.461673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6f5f9594-305c-4bd4-9959-2fbc3132e11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:48.469826 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1025/2056] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:48.474241 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b49a5b-8bb6-44db-b8fa-9cc7159c338c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:48.483838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b49a5b-8bb6-44db-b8fa-9cc7159c338c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:48.489270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca10953-0a90-4b78-bf00-fc37b294ace2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:48.489866 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca10953-0a90-4b78-bf00-fc37b294ace2 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:48.490487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca10953-0a90-4b78-bf00-fc37b294ace2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:48.490487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca10953-0a90-4b78-bf00-fc37b294ace2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:48.491066 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca10953-0a90-4b78-bf00-fc37b294ace2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:48.491367 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca10953-0a90-4b78-bf00-fc37b294ace2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:48.492961 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca10953-0a90-4b78-bf00-fc37b294ace2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:48.510293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1032/2057] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:48.513978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f9594-305c-4bd4-9959-2fbc3132e11b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:48.527090 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f9594-305c-4bd4-9959-2fbc3132e11b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:48.539694 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa2ffa-3048-4325-bc90-397883dd1e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:48.540431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa2ffa-3048-4325-bc90-397883dd1e6a admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:48.540803 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa2ffa-3048-4325-bc90-397883dd1e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:48.541589 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa2ffa-3048-4325-bc90-397883dd1e6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:48.542412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa2ffa-3048-4325-bc90-397883dd1e6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:48.544789 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa2ffa-3048-4325-bc90-397883dd1e6a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:48.606038 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca10953-0a90-4b78-bf00-fc37b294ace2 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:48.606334 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1026/2058] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:48 2022] POST /identity/v3/projects => generated 371 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:48.619628 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca10953-0a90-4b78-bf00-fc37b294ace2 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:48.667123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:48.689585 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1379d-24fa-4e6d-b3ae-062261d568c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:48.690232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1379d-24fa-4e6d-b3ae-062261d568c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:48.690553 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1379d-24fa-4e6d-b3ae-062261d568c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:48.690826 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1379d-24fa-4e6d-b3ae-062261d568c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:48.691456 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1379d-24fa-4e6d-b3ae-062261d568c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:48.694503 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1379d-24fa-4e6d-b3ae-062261d568c0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:48.724873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1033/2059] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:48 2022] POST /identity/v3/users => generated 286 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:48.729095 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa2ffa-3048-4325-bc90-397883dd1e6a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:48.741161 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa2ffa-3048-4325-bc90-397883dd1e6a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:48.756393 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb2c5b-93f1-4f0c-a5d1-e1343fc985b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:48.757010 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb2c5b-93f1-4f0c-a5d1-e1343fc985b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:48.757284 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb2c5b-93f1-4f0c-a5d1-e1343fc985b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:48.757546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb2c5b-93f1-4f0c-a5d1-e1343fc985b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:48.758217 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb2c5b-93f1-4f0c-a5d1-e1343fc985b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:48.759990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb2c5b-93f1-4f0c-a5d1-e1343fc985b7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:48.781177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1027/2060] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:48.782481 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1379d-24fa-4e6d-b3ae-062261d568c0 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:48.792528 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1379d-24fa-4e6d-b3ae-062261d568c0 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:48.798517 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161d049-8dfa-4ade-aa8f-661253d8e032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:48.799210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161d049-8dfa-4ade-aa8f-661253d8e032 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:48.799555 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161d049-8dfa-4ade-aa8f-661253d8e032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:48.799924 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161d049-8dfa-4ade-aa8f-661253d8e032 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:48.800637 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161d049-8dfa-4ade-aa8f-661253d8e032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:48.802518 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161d049-8dfa-4ade-aa8f-661253d8e032 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:48.832892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1034/2061] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:48.834795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb2c5b-93f1-4f0c-a5d1-e1343fc985b7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:48.848294 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb2c5b-93f1-4f0c-a5d1-e1343fc985b7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:48.855854 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7c8ed-07f7-4286-ab71-e316366e3b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:48.857055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7c8ed-07f7-4286-ab71-e316366e3b4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:48.857537 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7c8ed-07f7-4286-ab71-e316366e3b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:48.858007 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7c8ed-07f7-4286-ab71-e316366e3b4e admin admin] PATH_INFO: `/v3/projects/49fbf6d07542446c837b11b7927ab58e/users/5d816cc5eea842eabdd13180fe558603/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:48.886866 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7c8ed-07f7-4286-ab71-e316366e3b4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=49fbf6d07542446c837b11b7927ab58e, user_id=5d816cc5eea842eabdd13180fe558603, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:48.889885 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7c8ed-07f7-4286-ab71-e316366e3b4e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:48.905903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1028/2062] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:48 2022] PUT /identity/v3/projects/49fbf6d07542446c837b11b7927ab58e/users/5d816cc5eea842eabdd13180fe558603/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:48.917187 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f36a72-daa3-4715-a1ea-c18168cb738a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:48.917473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f36a72-daa3-4715-a1ea-c18168cb738a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:48.917767 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f36a72-daa3-4715-a1ea-c18168cb738a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:48.948495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d7f36a72-daa3-4715-a1ea-c18168cb738a None None] MFA Rules not processed for user `5d816cc5eea842eabdd13180fe558603`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:48.949811 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d7f36a72-daa3-4715-a1ea-c18168cb738a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:49.013835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1035/2063] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:49.025239 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7c8ed-07f7-4286-ab71-e316366e3b4e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:49.027601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c3a7c8ed-07f7-4286-ab71-e316366e3b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:49.084642 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7c8ed-07f7-4286-ab71-e316366e3b4e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:49.088361 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cd76a-f4b3-48b5-927f-b323a4ff2f1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:49.089001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cd76a-f4b3-48b5-927f-b323a4ff2f1e service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:49.089269 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cd76a-f4b3-48b5-927f-b323a4ff2f1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:49.089542 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cd76a-f4b3-48b5-927f-b323a4ff2f1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:49.097752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cd76a-f4b3-48b5-927f-b323a4ff2f1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:49.099205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cd76a-f4b3-48b5-927f-b323a4ff2f1e service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:49.107906 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1029/2064] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:49.132194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f36a72-daa3-4715-a1ea-c18168cb738a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:49.195509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f36a72-daa3-4715-a1ea-c18168cb738a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:49.201053 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fcf4b-0d72-4571-b793-55221b902914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:49.201681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fcf4b-0d72-4571-b793-55221b902914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:49.201961 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fcf4b-0d72-4571-b793-55221b902914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:49.202234 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fcf4b-0d72-4571-b793-55221b902914 admin admin] PATH_INFO: `/v3/users/5d816cc5eea842eabdd13180fe558603` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:49.203402 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fcf4b-0d72-4571-b793-55221b902914 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d816cc5eea842eabdd13180fe558603)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:49.205313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fcf4b-0d72-4571-b793-55221b902914 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:49.251879 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-023fcf4b-0d72-4571-b793-55221b902914 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d816cc5eea842eabdd13180fe558603'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:49.256388 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:49.261097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-023fcf4b-0d72-4571-b793-55221b902914 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d816cc5eea842eabdd13180fe558603'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:49.268721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-023fcf4b-0d72-4571-b793-55221b902914 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d816cc5eea842eabdd13180fe558603'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:49.269520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-023fcf4b-0d72-4571-b793-55221b902914 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d816cc5eea842eabdd13180fe558603'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:49.270185 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-023fcf4b-0d72-4571-b793-55221b902914 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d816cc5eea842eabdd13180fe558603'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:49.287131 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1036/2065] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:49 2022] DELETE /identity/v3/users/5d816cc5eea842eabdd13180fe558603 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:49.291527 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeaf596-67c5-4be8-a58c-56e1cb4247e9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:49.291816 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeaf596-67c5-4be8-a58c-56e1cb4247e9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:49.292093 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeaf596-67c5-4be8-a58c-56e1cb4247e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:49.310898 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-edeaf596-67c5-4be8-a58c-56e1cb4247e9 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:49.312090 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-edeaf596-67c5-4be8-a58c-56e1cb4247e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:49.371119 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1030/2066] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:49.381171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fcf4b-0d72-4571-b793-55221b902914 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:49.382614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-023fcf4b-0d72-4571-b793-55221b902914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:49.440408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fcf4b-0d72-4571-b793-55221b902914 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:49.444062 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4014ec-af2d-4842-87e5-a7db811fc33b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:49.444656 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4014ec-af2d-4842-87e5-a7db811fc33b service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:49.444933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4014ec-af2d-4842-87e5-a7db811fc33b service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:49.445360 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4014ec-af2d-4842-87e5-a7db811fc33b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:49.455045 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4014ec-af2d-4842-87e5-a7db811fc33b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:49.456460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4014ec-af2d-4842-87e5-a7db811fc33b service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:49.465717 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1037/2067] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:49.739669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeaf596-67c5-4be8-a58c-56e1cb4247e9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:49.741301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-edeaf596-67c5-4be8-a58c-56e1cb4247e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:49.751862 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:49.757011 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeaf596-67c5-4be8-a58c-56e1cb4247e9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:49.763709 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385aa2b0-07fa-4d5b-b825-6987c46a6b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:49.764747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385aa2b0-07fa-4d5b-b825-6987c46a6b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:49.765195 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385aa2b0-07fa-4d5b-b825-6987c46a6b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:49.765616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385aa2b0-07fa-4d5b-b825-6987c46a6b66 admin admin] PATH_INFO: `/v3/projects/49fbf6d07542446c837b11b7927ab58e` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:49.767032 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385aa2b0-07fa-4d5b-b825-6987c46a6b66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49fbf6d07542446c837b11b7927ab58e)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:49.769048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385aa2b0-07fa-4d5b-b825-6987c46a6b66 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:49.819491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-385aa2b0-07fa-4d5b-b825-6987c46a6b66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49fbf6d07542446c837b11b7927ab58e', 'request_id': 'req-385aa2b0-07fa-4d5b-b825-6987c46a6b66'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:49.825761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-385aa2b0-07fa-4d5b-b825-6987c46a6b66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49fbf6d07542446c837b11b7927ab58e', 'request_id': 'req-385aa2b0-07fa-4d5b-b825-6987c46a6b66'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:49.834606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-385aa2b0-07fa-4d5b-b825-6987c46a6b66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49fbf6d07542446c837b11b7927ab58e', 'request_id': 'req-385aa2b0-07fa-4d5b-b825-6987c46a6b66'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:49.847306 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-385aa2b0-07fa-4d5b-b825-6987c46a6b66 admin admin] The token cache is being invalidate because project 49fbf6d07542446c837b11b7927ab58e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:49.847596 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-385aa2b0-07fa-4d5b-b825-6987c46a6b66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:49.848262 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-385aa2b0-07fa-4d5b-b825-6987c46a6b66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:49.849849 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1031/2068] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:49 2022] DELETE /identity/v3/projects/49fbf6d07542446c837b11b7927ab58e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:49.855426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c999a9a-2367-4798-90e9-3ec52bd99002 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:49.855921 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c999a9a-2367-4798-90e9-3ec52bd99002 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:49.856130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c999a9a-2367-4798-90e9-3ec52bd99002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:49.876204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-1c999a9a-2367-4798-90e9-3ec52bd99002 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:49.877411 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1c999a9a-2367-4798-90e9-3ec52bd99002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:49.941347 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1038/2069] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:49.950535 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8e00e-2a21-4bc4-ab2d-d7ef270ceb6c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:49.950819 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8e00e-2a21-4bc4-ab2d-d7ef270ceb6c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:49.951101 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8e00e-2a21-4bc4-ab2d-d7ef270ceb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:49.971038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3af8e00e-2a21-4bc4-ab2d-d7ef270ceb6c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:49.972241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3af8e00e-2a21-4bc4-ab2d-d7ef270ceb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:49.983148 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1032/2070] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:49.987437 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c999a9a-2367-4798-90e9-3ec52bd99002 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:50.000282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c999a9a-2367-4798-90e9-3ec52bd99002 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:50.006078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c86b3-a1bf-4d39-938c-ad8a808d59ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:50.006762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c86b3-a1bf-4d39-938c-ad8a808d59ef admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:50.007045 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c86b3-a1bf-4d39-938c-ad8a808d59ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:50.007322 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c86b3-a1bf-4d39-938c-ad8a808d59ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:50.007990 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c86b3-a1bf-4d39-938c-ad8a808d59ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:50.008408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c86b3-a1bf-4d39-938c-ad8a808d59ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:50.010001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c86b3-a1bf-4d39-938c-ad8a808d59ef admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:50.023453 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1039/2071] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:50.028294 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8e00e-2a21-4bc4-ab2d-d7ef270ceb6c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:50.038737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8e00e-2a21-4bc4-ab2d-d7ef270ceb6c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:50.044452 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c17273-348a-4b9d-befc-943ae6fa2d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:50.044697 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c17273-348a-4b9d-befc-943ae6fa2d85 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:50.044979 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c17273-348a-4b9d-befc-943ae6fa2d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:50.045259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c17273-348a-4b9d-befc-943ae6fa2d85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:50.045994 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c17273-348a-4b9d-befc-943ae6fa2d85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:50.048093 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c17273-348a-4b9d-befc-943ae6fa2d85 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:50.077455 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1033/2072] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:50 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:50.083339 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c86b3-a1bf-4d39-938c-ad8a808d59ef admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:50.093851 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c86b3-a1bf-4d39-938c-ad8a808d59ef admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:50.147888 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6af53-28b7-4810-8dfc-a591738ebc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:50.148703 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6af53-28b7-4810-8dfc-a591738ebc17 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:50.148703 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6af53-28b7-4810-8dfc-a591738ebc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:50.148916 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6af53-28b7-4810-8dfc-a591738ebc17 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:50.149759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6af53-28b7-4810-8dfc-a591738ebc17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:50.152236 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6af53-28b7-4810-8dfc-a591738ebc17 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:50.161121 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:50.177285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1040/2073] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:50 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:50.181472 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c17273-348a-4b9d-befc-943ae6fa2d85 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:50.191837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c17273-348a-4b9d-befc-943ae6fa2d85 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:50.196368 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7953f-1fc0-4dda-9995-7f93b3a381c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:50.196947 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7953f-1fc0-4dda-9995-7f93b3a381c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:50.197223 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7953f-1fc0-4dda-9995-7f93b3a381c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:50.197488 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7953f-1fc0-4dda-9995-7f93b3a381c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:50.198641 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7953f-1fc0-4dda-9995-7f93b3a381c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:50.200210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7953f-1fc0-4dda-9995-7f93b3a381c7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:50.213120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1034/2074] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:50.214971 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6af53-28b7-4810-8dfc-a591738ebc17 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:50.224941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6af53-28b7-4810-8dfc-a591738ebc17 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:50.229945 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca818a-79a9-4d4e-b454-cc76d5f408a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:50.230583 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca818a-79a9-4d4e-b454-cc76d5f408a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:50.230857 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca818a-79a9-4d4e-b454-cc76d5f408a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:50.231127 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca818a-79a9-4d4e-b454-cc76d5f408a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:50.232017 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca818a-79a9-4d4e-b454-cc76d5f408a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:50.233590 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca818a-79a9-4d4e-b454-cc76d5f408a5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:50.250456 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1041/2075] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:50.252065 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7953f-1fc0-4dda-9995-7f93b3a381c7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:50.263176 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7953f-1fc0-4dda-9995-7f93b3a381c7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:50.267904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781f3ff-3eec-49ad-b4dc-4f2866f4596f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:50.268557 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781f3ff-3eec-49ad-b4dc-4f2866f4596f admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:50.269282 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781f3ff-3eec-49ad-b4dc-4f2866f4596f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:50.269565 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781f3ff-3eec-49ad-b4dc-4f2866f4596f admin admin] PATH_INFO: `/v3/projects/26d7688c90254acc819499404ab1d92f/users/0e36a1af5d294c0d88b19906e02b0d52/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:50.286205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781f3ff-3eec-49ad-b4dc-4f2866f4596f admin admin] RBAC: Authorizing `identity:create_grant(project_id=26d7688c90254acc819499404ab1d92f, user_id=0e36a1af5d294c0d88b19906e02b0d52, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:50.289799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781f3ff-3eec-49ad-b4dc-4f2866f4596f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:50.303314 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1035/2076] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:50 2022] PUT /identity/v3/projects/26d7688c90254acc819499404ab1d92f/users/0e36a1af5d294c0d88b19906e02b0d52/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:50.313791 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de597083-6b8f-4e61-ab87-699559423221 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:50.314490 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de597083-6b8f-4e61-ab87-699559423221 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:50.314769 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de597083-6b8f-4e61-ab87-699559423221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:50.359262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-de597083-6b8f-4e61-ab87-699559423221 None None] MFA Rules not processed for user `0e36a1af5d294c0d88b19906e02b0d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:50.361402 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-de597083-6b8f-4e61-ab87-699559423221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:50.450899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1042/2077] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:50.462118 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781f3ff-3eec-49ad-b4dc-4f2866f4596f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:50.464582 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e781f3ff-3eec-49ad-b4dc-4f2866f4596f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:50.528550 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781f3ff-3eec-49ad-b4dc-4f2866f4596f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:50.532520 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b1bb0-ab59-4c1f-bbd9-e943391c2ce7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:50.533131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b1bb0-ab59-4c1f-bbd9-e943391c2ce7 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:50.533409 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b1bb0-ab59-4c1f-bbd9-e943391c2ce7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:50.533680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b1bb0-ab59-4c1f-bbd9-e943391c2ce7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:50.544306 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b1bb0-ab59-4c1f-bbd9-e943391c2ce7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:50.546303 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b1bb0-ab59-4c1f-bbd9-e943391c2ce7 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:50.558091 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1036/2078] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:54.428532 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de597083-6b8f-4e61-ab87-699559423221 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:54.430057 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-de597083-6b8f-4e61-ab87-699559423221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:54.507828 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de597083-6b8f-4e61-ab87-699559423221 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:54.513034 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccc06e-7b7a-4295-91ba-7ee143763131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:54.513793 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccc06e-7b7a-4295-91ba-7ee143763131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:54.513927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccc06e-7b7a-4295-91ba-7ee143763131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:54.514203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccc06e-7b7a-4295-91ba-7ee143763131 admin admin] PATH_INFO: `/v3/users/e6840cd650924821a2681c4b0f80b63e` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:54.515379 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ccc06e-7b7a-4295-91ba-7ee143763131 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6840cd650924821a2681c4b0f80b63e)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:54.517357 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ccc06e-7b7a-4295-91ba-7ee143763131 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:54.590122 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-61ccc06e-7b7a-4295-91ba-7ee143763131 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6840cd650924821a2681c4b0f80b63e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:54.600407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-61ccc06e-7b7a-4295-91ba-7ee143763131 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6840cd650924821a2681c4b0f80b63e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:54.615839 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-61ccc06e-7b7a-4295-91ba-7ee143763131 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6840cd650924821a2681c4b0f80b63e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:54.616413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-61ccc06e-7b7a-4295-91ba-7ee143763131 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6840cd650924821a2681c4b0f80b63e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:54.617088 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-61ccc06e-7b7a-4295-91ba-7ee143763131 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6840cd650924821a2681c4b0f80b63e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:54.623482 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:54.645056 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1043/2079] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:54 2022] DELETE /identity/v3/users/e6840cd650924821a2681c4b0f80b63e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:54.646816 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0feaa22-329d-4ef7-b503-7ef03c937054 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:54.647676 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0feaa22-329d-4ef7-b503-7ef03c937054 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:54.648148 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0feaa22-329d-4ef7-b503-7ef03c937054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:54.672752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b0feaa22-329d-4ef7-b503-7ef03c937054 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:54.674468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b0feaa22-329d-4ef7-b503-7ef03c937054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:54.700930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:54.779644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1037/2080] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:54.821020 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccc06e-7b7a-4295-91ba-7ee143763131 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:54.823261 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-61ccc06e-7b7a-4295-91ba-7ee143763131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:54.915718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccc06e-7b7a-4295-91ba-7ee143763131 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:54.920609 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a631a4b-f016-493e-a859-7c008eeaa896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:54.921645 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a631a4b-f016-493e-a859-7c008eeaa896 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:54.922878 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a631a4b-f016-493e-a859-7c008eeaa896 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:54.923873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a631a4b-f016-493e-a859-7c008eeaa896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:54.938058 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a631a4b-f016-493e-a859-7c008eeaa896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:54.941160 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a631a4b-f016-493e-a859-7c008eeaa896 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:54.988803 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1044/2081] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:26:55.717763 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0feaa22-329d-4ef7-b503-7ef03c937054 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:55.719297 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b0feaa22-329d-4ef7-b503-7ef03c937054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:55.733257 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0feaa22-329d-4ef7-b503-7ef03c937054 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:55.738182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4e5ed-eeca-4442-a342-49c97395c721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:55.739049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4e5ed-eeca-4442-a342-49c97395c721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:55.739334 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4e5ed-eeca-4442-a342-49c97395c721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:55.739622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4e5ed-eeca-4442-a342-49c97395c721 admin admin] PATH_INFO: `/v3/projects/e96f7ae0385a42c79010a3bfae291da5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:55.740775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4e5ed-eeca-4442-a342-49c97395c721 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e96f7ae0385a42c79010a3bfae291da5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:55.742786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4e5ed-eeca-4442-a342-49c97395c721 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:55.817186 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-65c4e5ed-eeca-4442-a342-49c97395c721 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e96f7ae0385a42c79010a3bfae291da5', 'request_id': 'req-65c4e5ed-eeca-4442-a342-49c97395c721'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:55.823699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-65c4e5ed-eeca-4442-a342-49c97395c721 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e96f7ae0385a42c79010a3bfae291da5', 'request_id': 'req-65c4e5ed-eeca-4442-a342-49c97395c721'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:55.830691 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-65c4e5ed-eeca-4442-a342-49c97395c721 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e96f7ae0385a42c79010a3bfae291da5', 'request_id': 'req-65c4e5ed-eeca-4442-a342-49c97395c721'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:55.842065 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-65c4e5ed-eeca-4442-a342-49c97395c721 admin admin] The token cache is being invalidate because project e96f7ae0385a42c79010a3bfae291da5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:55.842452 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-65c4e5ed-eeca-4442-a342-49c97395c721 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:55.843154 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-65c4e5ed-eeca-4442-a342-49c97395c721 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:55.844644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1038/2082] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:55 2022] DELETE /identity/v3/projects/e96f7ae0385a42c79010a3bfae291da5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:55.848579 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a631a4b-f016-493e-a859-7c008eeaa896 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:55.866437 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a631a4b-f016-493e-a859-7c008eeaa896 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:55.929277 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bb628-b4eb-4285-8363-4c4294092372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:55.929905 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bb628-b4eb-4285-8363-4c4294092372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:55.930189 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bb628-b4eb-4285-8363-4c4294092372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:55.930548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bb628-b4eb-4285-8363-4c4294092372 admin admin] PATH_INFO: `/v3/users/5cfa8214286f49a8b41ccf341f73be62` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:55.931791 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10bb628-b4eb-4285-8363-4c4294092372 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cfa8214286f49a8b41ccf341f73be62)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:55.934309 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10bb628-b4eb-4285-8363-4c4294092372 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:55.983931 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e10bb628-b4eb-4285-8363-4c4294092372 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cfa8214286f49a8b41ccf341f73be62'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:55.990777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e10bb628-b4eb-4285-8363-4c4294092372 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cfa8214286f49a8b41ccf341f73be62'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:55.999565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e10bb628-b4eb-4285-8363-4c4294092372 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cfa8214286f49a8b41ccf341f73be62'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:56.000227 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e10bb628-b4eb-4285-8363-4c4294092372 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cfa8214286f49a8b41ccf341f73be62'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:56.000940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e10bb628-b4eb-4285-8363-4c4294092372 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cfa8214286f49a8b41ccf341f73be62'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:56.019162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1045/2083] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:55 2022] DELETE /identity/v3/users/5cfa8214286f49a8b41ccf341f73be62 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:56.366653 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4e5ed-eeca-4442-a342-49c97395c721 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:56.368185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-65c4e5ed-eeca-4442-a342-49c97395c721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:56.384775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4e5ed-eeca-4442-a342-49c97395c721 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:56.494417 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624abfc-afb3-457f-8f4f-c4675f7cfa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:56.495105 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624abfc-afb3-457f-8f4f-c4675f7cfa19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:56.495381 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624abfc-afb3-457f-8f4f-c4675f7cfa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:56.495735 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624abfc-afb3-457f-8f4f-c4675f7cfa19 admin admin] PATH_INFO: `/v3/projects/bd33b18093a349d594d63c892df79314` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:56.496774 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624abfc-afb3-457f-8f4f-c4675f7cfa19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd33b18093a349d594d63c892df79314)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:56.498786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624abfc-afb3-457f-8f4f-c4675f7cfa19 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:56.566069 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1624abfc-afb3-457f-8f4f-c4675f7cfa19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd33b18093a349d594d63c892df79314', 'request_id': 'req-1624abfc-afb3-457f-8f4f-c4675f7cfa19'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:56.568478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:56.572640 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1624abfc-afb3-457f-8f4f-c4675f7cfa19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd33b18093a349d594d63c892df79314', 'request_id': 'req-1624abfc-afb3-457f-8f4f-c4675f7cfa19'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:56.590408 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1624abfc-afb3-457f-8f4f-c4675f7cfa19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd33b18093a349d594d63c892df79314', 'request_id': 'req-1624abfc-afb3-457f-8f4f-c4675f7cfa19'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:56.593543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1624abfc-afb3-457f-8f4f-c4675f7cfa19 admin admin] The token cache is being invalidate because project bd33b18093a349d594d63c892df79314 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:56.593945 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1624abfc-afb3-457f-8f4f-c4675f7cfa19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:56.594697 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1624abfc-afb3-457f-8f4f-c4675f7cfa19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:56.596248 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1039/2084] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:56 2022] DELETE /identity/v3/projects/bd33b18093a349d594d63c892df79314 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:56.602024 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834c52c-4875-4d9b-af1c-16174ec0e80f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:56.602415 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834c52c-4875-4d9b-af1c-16174ec0e80f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:56.602716 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834c52c-4875-4d9b-af1c-16174ec0e80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:56.624283 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-3834c52c-4875-4d9b-af1c-16174ec0e80f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:56.624992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3834c52c-4875-4d9b-af1c-16174ec0e80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:56.629581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:56.705617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1046/2085] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:56.713584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d07097-3ae3-4930-8a7a-b61f6347907c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:56.714073 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d07097-3ae3-4930-8a7a-b61f6347907c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:56.714438 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d07097-3ae3-4930-8a7a-b61f6347907c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:56.733937 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-64d07097-3ae3-4930-8a7a-b61f6347907c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:56.735187 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-64d07097-3ae3-4930-8a7a-b61f6347907c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:56.745923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1040/2086] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:56.750302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834c52c-4875-4d9b-af1c-16174ec0e80f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:56.760163 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834c52c-4875-4d9b-af1c-16174ec0e80f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:56.766532 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e0977-4c19-48a2-ad25-98072d8a3db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:56.767579 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e0977-4c19-48a2-ad25-98072d8a3db5 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:56.767663 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e0977-4c19-48a2-ad25-98072d8a3db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:56.767939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e0977-4c19-48a2-ad25-98072d8a3db5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:56.768721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e0977-4c19-48a2-ad25-98072d8a3db5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:56.768898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e0977-4c19-48a2-ad25-98072d8a3db5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:56.770742 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e0977-4c19-48a2-ad25-98072d8a3db5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:56.783632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1047/2087] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:56.787613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d07097-3ae3-4930-8a7a-b61f6347907c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:56.799045 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d07097-3ae3-4930-8a7a-b61f6347907c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:56.806145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765bcdc-f29b-4db3-95f3-8f92dbc30e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:56.806798 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765bcdc-f29b-4db3-95f3-8f92dbc30e87 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:56.807727 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765bcdc-f29b-4db3-95f3-8f92dbc30e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:56.807943 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765bcdc-f29b-4db3-95f3-8f92dbc30e87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:56.808575 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765bcdc-f29b-4db3-95f3-8f92dbc30e87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:56.810605 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765bcdc-f29b-4db3-95f3-8f92dbc30e87 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:56.846953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1041/2088] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:56 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:56.851165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e0977-4c19-48a2-ad25-98072d8a3db5 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:56.864627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e0977-4c19-48a2-ad25-98072d8a3db5 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:56.931669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9015d4-360e-445c-8236-32fd0cd963ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:56.932814 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9015d4-360e-445c-8236-32fd0cd963ac admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:56.933095 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9015d4-360e-445c-8236-32fd0cd963ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:56.933399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9015d4-360e-445c-8236-32fd0cd963ac admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:56.934027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9015d4-360e-445c-8236-32fd0cd963ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:56.936035 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9015d4-360e-445c-8236-32fd0cd963ac admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:56.969124 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1048/2089] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:56 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:56.969219 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765bcdc-f29b-4db3-95f3-8f92dbc30e87 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:56.978407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765bcdc-f29b-4db3-95f3-8f92dbc30e87 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:56.984722 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70e1e3-edbb-4b88-98ac-dcff084850b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:56.985664 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70e1e3-edbb-4b88-98ac-dcff084850b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:56.986121 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70e1e3-edbb-4b88-98ac-dcff084850b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:56.986417 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70e1e3-edbb-4b88-98ac-dcff084850b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:56.987065 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e70e1e3-edbb-4b88-98ac-dcff084850b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:56.988683 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e70e1e3-edbb-4b88-98ac-dcff084850b8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:57.000629 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1042/2090] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:57.004514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9015d4-360e-445c-8236-32fd0cd963ac admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:57.015872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9015d4-360e-445c-8236-32fd0cd963ac admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:57.023291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325fb09-fa38-4275-a627-e2192560279a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:57.023977 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325fb09-fa38-4275-a627-e2192560279a admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:57.024326 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325fb09-fa38-4275-a627-e2192560279a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:57.024692 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325fb09-fa38-4275-a627-e2192560279a admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:57.025413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325fb09-fa38-4275-a627-e2192560279a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:57.027203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325fb09-fa38-4275-a627-e2192560279a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:57.039227 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1049/2091] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:57.043878 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70e1e3-edbb-4b88-98ac-dcff084850b8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:57.054473 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70e1e3-edbb-4b88-98ac-dcff084850b8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:57.059771 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad2386-1dec-4f6a-b1a4-61bebe36337c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:57.060416 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad2386-1dec-4f6a-b1a4-61bebe36337c admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:57.060724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad2386-1dec-4f6a-b1a4-61bebe36337c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:57.060996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad2386-1dec-4f6a-b1a4-61bebe36337c admin admin] PATH_INFO: `/v3/projects/1f7b210f225f450a938c664481e59f41/users/af568376ee6c4e63af1de7bdd0e7dadf/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:57.075861 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad2386-1dec-4f6a-b1a4-61bebe36337c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f7b210f225f450a938c664481e59f41, user_id=af568376ee6c4e63af1de7bdd0e7dadf, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:57.078563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad2386-1dec-4f6a-b1a4-61bebe36337c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:57.094238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1043/2092] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:57 2022] PUT /identity/v3/projects/1f7b210f225f450a938c664481e59f41/users/af568376ee6c4e63af1de7bdd0e7dadf/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:57.105892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84fbd8-836d-425b-8732-b5fcfeb8e7e9 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:57.106166 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84fbd8-836d-425b-8732-b5fcfeb8e7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:57.106517 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84fbd8-836d-425b-8732-b5fcfeb8e7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:57.137087 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-3d84fbd8-836d-425b-8732-b5fcfeb8e7e9 None None] MFA Rules not processed for user `af568376ee6c4e63af1de7bdd0e7dadf`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:57.138228 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3d84fbd8-836d-425b-8732-b5fcfeb8e7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:57.207656 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1050/2093] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:57.212237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad2386-1dec-4f6a-b1a4-61bebe36337c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:57.282394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad2386-1dec-4f6a-b1a4-61bebe36337c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:57.287138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8bd3f-ed9a-4f5b-91f3-0b2228eac293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:57.287737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8bd3f-ed9a-4f5b-91f3-0b2228eac293 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:57.288017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8bd3f-ed9a-4f5b-91f3-0b2228eac293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:57.288288 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8bd3f-ed9a-4f5b-91f3-0b2228eac293 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:57.289644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8bd3f-ed9a-4f5b-91f3-0b2228eac293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:57.292093 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8bd3f-ed9a-4f5b-91f3-0b2228eac293 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:57.321666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1044/2094] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:57 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:57.327170 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84fbd8-836d-425b-8732-b5fcfeb8e7e9 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:57.334685 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:57.340719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84fbd8-836d-425b-8732-b5fcfeb8e7e9 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:57.412977 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c6a55-6afc-4d2e-9d67-7ef4f8539528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:57.418601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c6a55-6afc-4d2e-9d67-7ef4f8539528 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:57.418895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c6a55-6afc-4d2e-9d67-7ef4f8539528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:57.419168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c6a55-6afc-4d2e-9d67-7ef4f8539528 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:57.419863 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c6a55-6afc-4d2e-9d67-7ef4f8539528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:57.422016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c6a55-6afc-4d2e-9d67-7ef4f8539528 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:57.447040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1051/2095] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:57 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:57.449036 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8bd3f-ed9a-4f5b-91f3-0b2228eac293 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:57.461572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8bd3f-ed9a-4f5b-91f3-0b2228eac293 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:57.466921 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7ffbe-4a67-4bf3-b1b3-5c12c2072c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:57.467595 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7ffbe-4a67-4bf3-b1b3-5c12c2072c59 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:57.467980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7ffbe-4a67-4bf3-b1b3-5c12c2072c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:57.468332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7ffbe-4a67-4bf3-b1b3-5c12c2072c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:57.469301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7ffbe-4a67-4bf3-b1b3-5c12c2072c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:57.471040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7ffbe-4a67-4bf3-b1b3-5c12c2072c59 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:57.483395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1045/2096] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:57.493773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c6a55-6afc-4d2e-9d67-7ef4f8539528 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:57.505065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c6a55-6afc-4d2e-9d67-7ef4f8539528 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:57.510411 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e583a2-4cf8-462f-b2ea-eca0333157f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:57.511088 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e583a2-4cf8-462f-b2ea-eca0333157f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:57.511359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e583a2-4cf8-462f-b2ea-eca0333157f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:57.511633 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e583a2-4cf8-462f-b2ea-eca0333157f1 admin admin] PATH_INFO: `/v3/projects/70d5aa1b4f6f4b449f267729ac5910cf/users/42c8f3cd1b234be0b052346e72569d29/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:57.531054 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e583a2-4cf8-462f-b2ea-eca0333157f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70d5aa1b4f6f4b449f267729ac5910cf, user_id=42c8f3cd1b234be0b052346e72569d29, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:57.533733 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e583a2-4cf8-462f-b2ea-eca0333157f1 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:57.550291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1052/2097] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:57 2022] PUT /identity/v3/projects/70d5aa1b4f6f4b449f267729ac5910cf/users/42c8f3cd1b234be0b052346e72569d29/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:57.555368 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7ffbe-4a67-4bf3-b1b3-5c12c2072c59 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:57.567515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7ffbe-4a67-4bf3-b1b3-5c12c2072c59 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:57.629653 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:57.641963 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ca878-1213-4600-9655-285345668d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:57.643234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ca878-1213-4600-9655-285345668d96 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:57.643942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ca878-1213-4600-9655-285345668d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:57.645008 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ca878-1213-4600-9655-285345668d96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:57.645846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ca878-1213-4600-9655-285345668d96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:57.650455 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ca878-1213-4600-9655-285345668d96 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:57.665015 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1046/2098] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:57.667706 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e583a2-4cf8-462f-b2ea-eca0333157f1 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:57.679992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e583a2-4cf8-462f-b2ea-eca0333157f1 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:57.687400 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26769159-bc2a-4fd2-8f80-f752e0e8ab12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:57.688176 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26769159-bc2a-4fd2-8f80-f752e0e8ab12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:57.688551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26769159-bc2a-4fd2-8f80-f752e0e8ab12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:57.688923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26769159-bc2a-4fd2-8f80-f752e0e8ab12 admin admin] PATH_INFO: `/v3/projects/70d5aa1b4f6f4b449f267729ac5910cf/users/42c8f3cd1b234be0b052346e72569d29/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:57.691639 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26769159-bc2a-4fd2-8f80-f752e0e8ab12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70d5aa1b4f6f4b449f267729ac5910cf, user_id=42c8f3cd1b234be0b052346e72569d29, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:57.695184 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26769159-bc2a-4fd2-8f80-f752e0e8ab12 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:57.709862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1053/2099] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:57 2022] PUT /identity/v3/projects/70d5aa1b4f6f4b449f267729ac5910cf/users/42c8f3cd1b234be0b052346e72569d29/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:57.720666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa88c0-4b32-49a3-ad1b-43bec7ed4dc2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:57.721182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa88c0-4b32-49a3-ad1b-43bec7ed4dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:57.721573 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa88c0-4b32-49a3-ad1b-43bec7ed4dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:57.774919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-edaa88c0-4b32-49a3-ad1b-43bec7ed4dc2 None None] MFA Rules not processed for user `42c8f3cd1b234be0b052346e72569d29`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:57.777210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-edaa88c0-4b32-49a3-ad1b-43bec7ed4dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:57.858745 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1047/2100] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:26:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:57.869439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26769159-bc2a-4fd2-8f80-f752e0e8ab12 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:57.871275 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-26769159-bc2a-4fd2-8f80-f752e0e8ab12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:57.927366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26769159-bc2a-4fd2-8f80-f752e0e8ab12 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:57.930920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c311a8-0397-4c77-95bd-04a6d45c82ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:57.931511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c311a8-0397-4c77-95bd-04a6d45c82ae service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:57.931789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c311a8-0397-4c77-95bd-04a6d45c82ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:57.932084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c311a8-0397-4c77-95bd-04a6d45c82ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:57.987280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c311a8-0397-4c77-95bd-04a6d45c82ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:57.988712 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c311a8-0397-4c77-95bd-04a6d45c82ae service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:57.997460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1054/2101] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:04.701737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa88c0-4b32-49a3-ad1b-43bec7ed4dc2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:04.712135 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa88c0-4b32-49a3-ad1b-43bec7ed4dc2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:04.717006 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee0e1c-49a6-43dc-8289-dd863b3c08b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:04.717865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee0e1c-49a6-43dc-8289-dd863b3c08b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:04.718152 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee0e1c-49a6-43dc-8289-dd863b3c08b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:04.718453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee0e1c-49a6-43dc-8289-dd863b3c08b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:04.727991 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee0e1c-49a6-43dc-8289-dd863b3c08b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:04.729486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee0e1c-49a6-43dc-8289-dd863b3c08b6 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:04.742088 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1048/2102] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:05.326477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c311a8-0397-4c77-95bd-04a6d45c82ae service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:05.327898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f0c311a8-0397-4c77-95bd-04a6d45c82ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:05.341068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:05.393022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c311a8-0397-4c77-95bd-04a6d45c82ae service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:05.398231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673791c4-35ed-469b-93ff-cf1ae1b65c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:05.398876 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673791c4-35ed-469b-93ff-cf1ae1b65c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:05.399150 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673791c4-35ed-469b-93ff-cf1ae1b65c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:05.399419 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673791c4-35ed-469b-93ff-cf1ae1b65c30 admin admin] PATH_INFO: `/v3/users/0e36a1af5d294c0d88b19906e02b0d52` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:05.400527 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673791c4-35ed-469b-93ff-cf1ae1b65c30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e36a1af5d294c0d88b19906e02b0d52)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:05.402545 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673791c4-35ed-469b-93ff-cf1ae1b65c30 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:05.451991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-673791c4-35ed-469b-93ff-cf1ae1b65c30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e36a1af5d294c0d88b19906e02b0d52'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:05.459266 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-673791c4-35ed-469b-93ff-cf1ae1b65c30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e36a1af5d294c0d88b19906e02b0d52'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:05.466414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-673791c4-35ed-469b-93ff-cf1ae1b65c30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e36a1af5d294c0d88b19906e02b0d52'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:05.467041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-673791c4-35ed-469b-93ff-cf1ae1b65c30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e36a1af5d294c0d88b19906e02b0d52'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:05.467659 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-673791c4-35ed-469b-93ff-cf1ae1b65c30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e36a1af5d294c0d88b19906e02b0d52'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:05.484744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1055/2103] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:05 2022] DELETE /identity/v3/users/0e36a1af5d294c0d88b19906e02b0d52 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:05.490681 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa71113-2f4a-4a08-9d14-ca0710129d77 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:05.490973 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa71113-2f4a-4a08-9d14-ca0710129d77 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:05.491252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa71113-2f4a-4a08-9d14-ca0710129d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:05.510739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8aa71113-2f4a-4a08-9d14-ca0710129d77 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:05.511946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8aa71113-2f4a-4a08-9d14-ca0710129d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:05.572655 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1049/2104] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:05.582871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673791c4-35ed-469b-93ff-cf1ae1b65c30 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:05.584367 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-673791c4-35ed-469b-93ff-cf1ae1b65c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:05.648621 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673791c4-35ed-469b-93ff-cf1ae1b65c30 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:05.652901 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81918e5a-e76c-4019-8985-c389b2253f59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:05.653533 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81918e5a-e76c-4019-8985-c389b2253f59 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:05.653814 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81918e5a-e76c-4019-8985-c389b2253f59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:05.654088 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81918e5a-e76c-4019-8985-c389b2253f59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:05.664060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81918e5a-e76c-4019-8985-c389b2253f59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:05.665524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81918e5a-e76c-4019-8985-c389b2253f59 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:05.676960 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1056/2105] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:06.032575 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa71113-2f4a-4a08-9d14-ca0710129d77 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:06.034725 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8aa71113-2f4a-4a08-9d14-ca0710129d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:06.047200 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa71113-2f4a-4a08-9d14-ca0710129d77 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:06.052069 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9bbfb-b0a0-4144-a74e-bae5bf39babe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:06.052799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9bbfb-b0a0-4144-a74e-bae5bf39babe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:06.053136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9bbfb-b0a0-4144-a74e-bae5bf39babe admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:06.053466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9bbfb-b0a0-4144-a74e-bae5bf39babe admin admin] PATH_INFO: `/v3/projects/26d7688c90254acc819499404ab1d92f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:06.054603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9bbfb-b0a0-4144-a74e-bae5bf39babe admin admin] RBAC: Authorizing `identity:delete_project(project_id=26d7688c90254acc819499404ab1d92f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:06.056512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9bbfb-b0a0-4144-a74e-bae5bf39babe admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:06.106236 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-23b9bbfb-b0a0-4144-a74e-bae5bf39babe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26d7688c90254acc819499404ab1d92f', 'request_id': 'req-23b9bbfb-b0a0-4144-a74e-bae5bf39babe'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:06.110062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:06.112345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-23b9bbfb-b0a0-4144-a74e-bae5bf39babe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26d7688c90254acc819499404ab1d92f', 'request_id': 'req-23b9bbfb-b0a0-4144-a74e-bae5bf39babe'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:06.121184 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-23b9bbfb-b0a0-4144-a74e-bae5bf39babe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26d7688c90254acc819499404ab1d92f', 'request_id': 'req-23b9bbfb-b0a0-4144-a74e-bae5bf39babe'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:06.137710 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-23b9bbfb-b0a0-4144-a74e-bae5bf39babe admin admin] The token cache is being invalidate because project 26d7688c90254acc819499404ab1d92f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:06.138075 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-23b9bbfb-b0a0-4144-a74e-bae5bf39babe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:06.138918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-23b9bbfb-b0a0-4144-a74e-bae5bf39babe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:06.140684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1050/2106] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:06 2022] DELETE /identity/v3/projects/26d7688c90254acc819499404ab1d92f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:15.671956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=74,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:16.133467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=2,keystone:SELECT=5,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:40.313719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd978cd-c055-4aea-913d-90dbb9622b61 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:40.314598 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd978cd-c055-4aea-913d-90dbb9622b61 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:40.314598 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd978cd-c055-4aea-913d-90dbb9622b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:40.337811 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ddd978cd-c055-4aea-913d-90dbb9622b61 None None] MFA Rules not processed for user `6916c2afef12490f921d99c5047b4dce`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:40.339666 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ddd978cd-c055-4aea-913d-90dbb9622b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:40.422336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1057/2107] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:41.034857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9bbfb-b0a0-4144-a74e-bae5bf39babe admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:41.036509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-23b9bbfb-b0a0-4144-a74e-bae5bf39babe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:41.104664 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9bbfb-b0a0-4144-a74e-bae5bf39babe admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:41.109801 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16214c-6d08-45d5-b997-859efb5ffd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:41.110472 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16214c-6d08-45d5-b997-859efb5ffd3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:41.110752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16214c-6d08-45d5-b997-859efb5ffd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:41.111028 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16214c-6d08-45d5-b997-859efb5ffd3f admin admin] PATH_INFO: `/v3/users/af568376ee6c4e63af1de7bdd0e7dadf` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:41.112360 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16214c-6d08-45d5-b997-859efb5ffd3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=af568376ee6c4e63af1de7bdd0e7dadf)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:41.114312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16214c-6d08-45d5-b997-859efb5ffd3f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:41.166201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1f16214c-6d08-45d5-b997-859efb5ffd3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af568376ee6c4e63af1de7bdd0e7dadf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:41.180308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1f16214c-6d08-45d5-b997-859efb5ffd3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af568376ee6c4e63af1de7bdd0e7dadf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:41.190439 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1f16214c-6d08-45d5-b997-859efb5ffd3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af568376ee6c4e63af1de7bdd0e7dadf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:41.191434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1f16214c-6d08-45d5-b997-859efb5ffd3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af568376ee6c4e63af1de7bdd0e7dadf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:41.192306 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1f16214c-6d08-45d5-b997-859efb5ffd3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af568376ee6c4e63af1de7bdd0e7dadf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:41.214159 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1051/2108] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:41 2022] DELETE /identity/v3/users/af568376ee6c4e63af1de7bdd0e7dadf => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:27:41.219408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed10772-9aa1-47b8-89b3-1c7ceabe15b0 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:41.219868 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed10772-9aa1-47b8-89b3-1c7ceabe15b0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:41.220150 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed10772-9aa1-47b8-89b3-1c7ceabe15b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:41.248885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-1ed10772-9aa1-47b8-89b3-1c7ceabe15b0 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:41.250135 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1ed10772-9aa1-47b8-89b3-1c7ceabe15b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:41.334422 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1058/2109] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:41.342505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16214c-6d08-45d5-b997-859efb5ffd3f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:41.344041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1f16214c-6d08-45d5-b997-859efb5ffd3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:41.417529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16214c-6d08-45d5-b997-859efb5ffd3f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:41.424319 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf177e7b-d150-446d-9135-92424ad22522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:41.426169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf177e7b-d150-446d-9135-92424ad22522 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:41.426544 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf177e7b-d150-446d-9135-92424ad22522 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:41.426816 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf177e7b-d150-446d-9135-92424ad22522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:41.439092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf177e7b-d150-446d-9135-92424ad22522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:41.451615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf177e7b-d150-446d-9135-92424ad22522 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:41.463806 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1052/2110] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:41.776738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed10772-9aa1-47b8-89b3-1c7ceabe15b0 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:41.778461 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1ed10772-9aa1-47b8-89b3-1c7ceabe15b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:41.794909 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed10772-9aa1-47b8-89b3-1c7ceabe15b0 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:41.801847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d34cc2-6093-477f-b4dd-04b8bf26e0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:41.802548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d34cc2-6093-477f-b4dd-04b8bf26e0ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:41.802830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d34cc2-6093-477f-b4dd-04b8bf26e0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:41.803170 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d34cc2-6093-477f-b4dd-04b8bf26e0ac admin admin] PATH_INFO: `/v3/projects/1f7b210f225f450a938c664481e59f41` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:41.804299 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d34cc2-6093-477f-b4dd-04b8bf26e0ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f7b210f225f450a938c664481e59f41)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:41.806122 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d34cc2-6093-477f-b4dd-04b8bf26e0ac admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:41.856650 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-03d34cc2-6093-477f-b4dd-04b8bf26e0ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f7b210f225f450a938c664481e59f41', 'request_id': 'req-03d34cc2-6093-477f-b4dd-04b8bf26e0ac'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:41.862053 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-03d34cc2-6093-477f-b4dd-04b8bf26e0ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f7b210f225f450a938c664481e59f41', 'request_id': 'req-03d34cc2-6093-477f-b4dd-04b8bf26e0ac'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:41.871028 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-03d34cc2-6093-477f-b4dd-04b8bf26e0ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f7b210f225f450a938c664481e59f41', 'request_id': 'req-03d34cc2-6093-477f-b4dd-04b8bf26e0ac'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:41.890171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-03d34cc2-6093-477f-b4dd-04b8bf26e0ac admin admin] The token cache is being invalidate because project 1f7b210f225f450a938c664481e59f41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:41.890400 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-03d34cc2-6093-477f-b4dd-04b8bf26e0ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:41.891049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-03d34cc2-6093-477f-b4dd-04b8bf26e0ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:41.892556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1059/2111] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:41 2022] DELETE /identity/v3/projects/1f7b210f225f450a938c664481e59f41 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:41.896676 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf177e7b-d150-446d-9135-92424ad22522 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:41.907072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf177e7b-d150-446d-9135-92424ad22522 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:41.926172 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:41.968548 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04441a8-6b06-4169-82dd-b371f1d89637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:41.969280 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04441a8-6b06-4169-82dd-b371f1d89637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:41.969559 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04441a8-6b06-4169-82dd-b371f1d89637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:41.969989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04441a8-6b06-4169-82dd-b371f1d89637 admin admin] PATH_INFO: `/v3/users/42c8f3cd1b234be0b052346e72569d29` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:41.971199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04441a8-6b06-4169-82dd-b371f1d89637 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42c8f3cd1b234be0b052346e72569d29)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:41.973145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04441a8-6b06-4169-82dd-b371f1d89637 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:42.023717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e04441a8-6b06-4169-82dd-b371f1d89637 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42c8f3cd1b234be0b052346e72569d29'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:42.032457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e04441a8-6b06-4169-82dd-b371f1d89637 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42c8f3cd1b234be0b052346e72569d29'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:42.040105 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e04441a8-6b06-4169-82dd-b371f1d89637 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42c8f3cd1b234be0b052346e72569d29'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:42.041180 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e04441a8-6b06-4169-82dd-b371f1d89637 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42c8f3cd1b234be0b052346e72569d29'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:42.041794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e04441a8-6b06-4169-82dd-b371f1d89637 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42c8f3cd1b234be0b052346e72569d29'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:42.060698 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1053/2112] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:41 2022] DELETE /identity/v3/users/42c8f3cd1b234be0b052346e72569d29 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:42.396329 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d34cc2-6093-477f-b4dd-04b8bf26e0ac admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:42.397958 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-03d34cc2-6093-477f-b4dd-04b8bf26e0ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:42.419635 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d34cc2-6093-477f-b4dd-04b8bf26e0ac admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:42.452952 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:42.496031 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62f178-f413-4f08-b82a-7c35c6f61ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:42.497080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62f178-f413-4f08-b82a-7c35c6f61ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:42.497501 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62f178-f413-4f08-b82a-7c35c6f61ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:42.497906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62f178-f413-4f08-b82a-7c35c6f61ad1 admin admin] PATH_INFO: `/v3/projects/70d5aa1b4f6f4b449f267729ac5910cf` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:42.499617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62f178-f413-4f08-b82a-7c35c6f61ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70d5aa1b4f6f4b449f267729ac5910cf)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:42.503308 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62f178-f413-4f08-b82a-7c35c6f61ad1 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:42.552923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0d62f178-f413-4f08-b82a-7c35c6f61ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70d5aa1b4f6f4b449f267729ac5910cf', 'request_id': 'req-0d62f178-f413-4f08-b82a-7c35c6f61ad1'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:42.558958 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0d62f178-f413-4f08-b82a-7c35c6f61ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70d5aa1b4f6f4b449f267729ac5910cf', 'request_id': 'req-0d62f178-f413-4f08-b82a-7c35c6f61ad1'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:42.567954 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0d62f178-f413-4f08-b82a-7c35c6f61ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70d5aa1b4f6f4b449f267729ac5910cf', 'request_id': 'req-0d62f178-f413-4f08-b82a-7c35c6f61ad1'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:42.581675 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0d62f178-f413-4f08-b82a-7c35c6f61ad1 admin admin] The token cache is being invalidate because project 70d5aa1b4f6f4b449f267729ac5910cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:42.581966 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0d62f178-f413-4f08-b82a-7c35c6f61ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:42.582666 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0d62f178-f413-4f08-b82a-7c35c6f61ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:42.584267 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1060/2113] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:42 2022] DELETE /identity/v3/projects/70d5aa1b4f6f4b449f267729ac5910cf => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:42.589149 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f5eaf-4ca7-44d7-8433-5d199edad05f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:42.589433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f5eaf-4ca7-44d7-8433-5d199edad05f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:42.589763 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f5eaf-4ca7-44d7-8433-5d199edad05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:42.617923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-855f5eaf-4ca7-44d7-8433-5d199edad05f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:42.619163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-855f5eaf-4ca7-44d7-8433-5d199edad05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:42.793891 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1054/2114] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:42.796694 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb9b05-3bc9-4aa4-80f0-24eb6c276f28 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:42.796951 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb9b05-3bc9-4aa4-80f0-24eb6c276f28 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:42.797232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb9b05-3bc9-4aa4-80f0-24eb6c276f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:42.825208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d1eb9b05-3bc9-4aa4-80f0-24eb6c276f28 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:42.826574 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d1eb9b05-3bc9-4aa4-80f0-24eb6c276f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:42.837478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1061/2115] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:42.844987 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f5eaf-4ca7-44d7-8433-5d199edad05f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:42.856321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f5eaf-4ca7-44d7-8433-5d199edad05f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:42.872895 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45f3ba-5e5c-4806-80fe-62913c9db2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:42.873541 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45f3ba-5e5c-4806-80fe-62913c9db2c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:42.873816 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45f3ba-5e5c-4806-80fe-62913c9db2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:42.874084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45f3ba-5e5c-4806-80fe-62913c9db2c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:42.875332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45f3ba-5e5c-4806-80fe-62913c9db2c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:27:42.875644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45f3ba-5e5c-4806-80fe-62913c9db2c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:42.877574 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45f3ba-5e5c-4806-80fe-62913c9db2c1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:42.897519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb9b05-3bc9-4aa4-80f0-24eb6c276f28 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:42.907620 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1055/2116] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:27:42.912908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb9b05-3bc9-4aa4-80f0-24eb6c276f28 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:42.922332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac52312-3ecc-481f-923d-5b0efa51e2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:42.922721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac52312-3ecc-481f-923d-5b0efa51e2c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:42.923002 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac52312-3ecc-481f-923d-5b0efa51e2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:42.923276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac52312-3ecc-481f-923d-5b0efa51e2c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:42.923966 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac52312-3ecc-481f-923d-5b0efa51e2c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:42.930904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac52312-3ecc-481f-923d-5b0efa51e2c6 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:42.971356 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1062/2117] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:42 2022] POST /identity/v3/projects => generated 387 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:27:42.975911 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45f3ba-5e5c-4806-80fe-62913c9db2c1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:42.987817 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45f3ba-5e5c-4806-80fe-62913c9db2c1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:43.022826 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:43.071313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87371841-e44b-41a0-965f-202fe8e6b6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:43.071977 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87371841-e44b-41a0-965f-202fe8e6b6c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:43.072406 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87371841-e44b-41a0-965f-202fe8e6b6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:43.072750 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87371841-e44b-41a0-965f-202fe8e6b6c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:43.074400 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87371841-e44b-41a0-965f-202fe8e6b6c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:43.076614 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87371841-e44b-41a0-965f-202fe8e6b6c1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:43.109059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1056/2118] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:42 2022] POST /identity/v3/users => generated 294 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:27:43.109701 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac52312-3ecc-481f-923d-5b0efa51e2c6 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:43.123668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac52312-3ecc-481f-923d-5b0efa51e2c6 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:43.129268 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3112e3-85f6-49c9-8e65-c05ac341917b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:43.129953 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3112e3-85f6-49c9-8e65-c05ac341917b admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:43.130442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3112e3-85f6-49c9-8e65-c05ac341917b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:43.130779 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3112e3-85f6-49c9-8e65-c05ac341917b admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:43.131505 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3112e3-85f6-49c9-8e65-c05ac341917b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:43.133315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3112e3-85f6-49c9-8e65-c05ac341917b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:43.146498 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1063/2119] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:27:43.149048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87371841-e44b-41a0-965f-202fe8e6b6c1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:43.160243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87371841-e44b-41a0-965f-202fe8e6b6c1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:43.165389 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce896c-f160-4551-9d57-7b5cf49f3153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:43.166087 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce896c-f160-4551-9d57-7b5cf49f3153 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:43.166469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce896c-f160-4551-9d57-7b5cf49f3153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:43.166841 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce896c-f160-4551-9d57-7b5cf49f3153 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:43.167549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce896c-f160-4551-9d57-7b5cf49f3153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:43.169386 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce896c-f160-4551-9d57-7b5cf49f3153 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:43.205569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1057/2120] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:43.206570 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3112e3-85f6-49c9-8e65-c05ac341917b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:43.220803 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3112e3-85f6-49c9-8e65-c05ac341917b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:43.246269 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c1908-abd4-465c-8dc7-6d4eb6ac55e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:43.246269 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c1908-abd4-465c-8dc7-6d4eb6ac55e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:43.246269 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c1908-abd4-465c-8dc7-6d4eb6ac55e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:43.246269 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c1908-abd4-465c-8dc7-6d4eb6ac55e3 admin admin] PATH_INFO: `/v3/projects/7e8274dd2d5b4d028e805d59f4ac8a75/users/9563e12eb09e4970b47075837a63128b/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:43.275429 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c1908-abd4-465c-8dc7-6d4eb6ac55e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e8274dd2d5b4d028e805d59f4ac8a75, user_id=9563e12eb09e4970b47075837a63128b, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:43.278174 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c1908-abd4-465c-8dc7-6d4eb6ac55e3 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:43.300535 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1064/2121] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:43 2022] PUT /identity/v3/projects/7e8274dd2d5b4d028e805d59f4ac8a75/users/9563e12eb09e4970b47075837a63128b/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:43.311557 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb11bcf-c8b5-4a2e-b496-03cd0f2a0edc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:43.311952 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb11bcf-c8b5-4a2e-b496-03cd0f2a0edc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:43.312252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb11bcf-c8b5-4a2e-b496-03cd0f2a0edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:43.350827 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-afb11bcf-c8b5-4a2e-b496-03cd0f2a0edc None None] MFA Rules not processed for user `9563e12eb09e4970b47075837a63128b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:43.352155 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-afb11bcf-c8b5-4a2e-b496-03cd0f2a0edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:43.444332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1058/2122] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:43.446351 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c1908-abd4-465c-8dc7-6d4eb6ac55e3 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:43.534962 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:43.541416 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c1908-abd4-465c-8dc7-6d4eb6ac55e3 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:43.548565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecea3c-587d-49de-9af2-0e0a54917e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:43.549263 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecea3c-587d-49de-9af2-0e0a54917e0c admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:43.549881 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecea3c-587d-49de-9af2-0e0a54917e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:43.550515 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecea3c-587d-49de-9af2-0e0a54917e0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:43.551433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ecea3c-587d-49de-9af2-0e0a54917e0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:43.553619 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ecea3c-587d-49de-9af2-0e0a54917e0c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:43.588123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1065/2123] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:43 2022] POST /identity/v3/projects => generated 387 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:43.592062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb11bcf-c8b5-4a2e-b496-03cd0f2a0edc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:43.603907 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb11bcf-c8b5-4a2e-b496-03cd0f2a0edc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:43.674088 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38bcc4-3c08-4d8d-829e-9375254e3183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:43.674639 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38bcc4-3c08-4d8d-829e-9375254e3183 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:43.675426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38bcc4-3c08-4d8d-829e-9375254e3183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:43.675705 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38bcc4-3c08-4d8d-829e-9375254e3183 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:43.676342 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38bcc4-3c08-4d8d-829e-9375254e3183 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:43.679356 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38bcc4-3c08-4d8d-829e-9375254e3183 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:43.704710 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1059/2124] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:43 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:27:43.708528 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecea3c-587d-49de-9af2-0e0a54917e0c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:43.720699 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecea3c-587d-49de-9af2-0e0a54917e0c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:43.727890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b93eb-4996-423b-8e26-2139b27d2e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:43.728504 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b93eb-4996-423b-8e26-2139b27d2e79 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:43.728780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b93eb-4996-423b-8e26-2139b27d2e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:43.729659 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b93eb-4996-423b-8e26-2139b27d2e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:43.731036 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b93eb-4996-423b-8e26-2139b27d2e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:43.732658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b93eb-4996-423b-8e26-2139b27d2e79 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:43.743904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1066/2125] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:43.748698 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38bcc4-3c08-4d8d-829e-9375254e3183 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:43.759399 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38bcc4-3c08-4d8d-829e-9375254e3183 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:43.764284 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b0f5a-d467-4503-88f0-135d458c9aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:43.765005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b0f5a-d467-4503-88f0-135d458c9aae admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:43.765293 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b0f5a-d467-4503-88f0-135d458c9aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:43.765570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b0f5a-d467-4503-88f0-135d458c9aae admin admin] PATH_INFO: `/v3/projects/9bac15c2479547768ced270aef0d782d/users/1f5d9026aa284cbe9a23fd3c30e6361f/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:43.785752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b0f5a-d467-4503-88f0-135d458c9aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bac15c2479547768ced270aef0d782d, user_id=1f5d9026aa284cbe9a23fd3c30e6361f, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:43.788664 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b0f5a-d467-4503-88f0-135d458c9aae admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:43.809433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1060/2126] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:43 2022] PUT /identity/v3/projects/9bac15c2479547768ced270aef0d782d/users/1f5d9026aa284cbe9a23fd3c30e6361f/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:43.814715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b93eb-4996-423b-8e26-2139b27d2e79 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:43.830590 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b93eb-4996-423b-8e26-2139b27d2e79 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:44.035024 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cfd2f-2d0b-4558-8289-c1c0cc369001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:44.035782 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cfd2f-2d0b-4558-8289-c1c0cc369001 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:44.036124 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cfd2f-2d0b-4558-8289-c1c0cc369001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:44.038200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cfd2f-2d0b-4558-8289-c1c0cc369001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:44.038978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cfd2f-2d0b-4558-8289-c1c0cc369001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:44.047465 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cfd2f-2d0b-4558-8289-c1c0cc369001 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:44.084837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1067/2127] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:44.090282 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b0f5a-d467-4503-88f0-135d458c9aae admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:44.098576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:44.112936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b0f5a-d467-4503-88f0-135d458c9aae admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:44.128238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d122b0-9cf3-4757-a2a2-ff2dcd0c3aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:44.129038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d122b0-9cf3-4757-a2a2-ff2dcd0c3aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:44.129379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d122b0-9cf3-4757-a2a2-ff2dcd0c3aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:44.129725 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d122b0-9cf3-4757-a2a2-ff2dcd0c3aa7 admin admin] PATH_INFO: `/v3/projects/9bac15c2479547768ced270aef0d782d/users/1f5d9026aa284cbe9a23fd3c30e6361f/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:44.131794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d122b0-9cf3-4757-a2a2-ff2dcd0c3aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bac15c2479547768ced270aef0d782d, user_id=1f5d9026aa284cbe9a23fd3c30e6361f, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:44.135184 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d122b0-9cf3-4757-a2a2-ff2dcd0c3aa7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:44.173162 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1061/2128] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:44 2022] PUT /identity/v3/projects/9bac15c2479547768ced270aef0d782d/users/1f5d9026aa284cbe9a23fd3c30e6361f/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:44.182562 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5b352-5dc5-4260-8d89-dfa4a05b5a86 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:44.182950 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5b352-5dc5-4260-8d89-dfa4a05b5a86 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:44.183293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5b352-5dc5-4260-8d89-dfa4a05b5a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:44.216847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-e1d5b352-5dc5-4260-8d89-dfa4a05b5a86 None None] MFA Rules not processed for user `1f5d9026aa284cbe9a23fd3c30e6361f`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:44.218662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e1d5b352-5dc5-4260-8d89-dfa4a05b5a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:44.226921 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0029d17-7522-4dee-82a9-abdb6145753f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:44.227734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0029d17-7522-4dee-82a9-abdb6145753f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:44.228639 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0029d17-7522-4dee-82a9-abdb6145753f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:44.306029 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b0029d17-7522-4dee-82a9-abdb6145753f None None] MFA Rules not processed for user `b88ccd3e5c4c48d3a27f4eaa7874e28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:44.308040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b0029d17-7522-4dee-82a9-abdb6145753f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:44.336751 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1068/2129] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:44 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:44.357352 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5b352-5dc5-4260-8d89-dfa4a05b5a86 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:44.359042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e1d5b352-5dc5-4260-8d89-dfa4a05b5a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:44.436016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1062/2130] 10.222.0.39 () {58 vars in 1009 bytes} [Mon Aug 8 02:27:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:44.467347 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5b352-5dc5-4260-8d89-dfa4a05b5a86 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:44.478873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6017041-9702-4a19-90f1-725ceb931821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:44.479575 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6017041-9702-4a19-90f1-725ceb931821 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:44.479858 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6017041-9702-4a19-90f1-725ceb931821 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:44.480132 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6017041-9702-4a19-90f1-725ceb931821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:44.491945 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:44.493261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0029d17-7522-4dee-82a9-abdb6145753f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:44.494991 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b0029d17-7522-4dee-82a9-abdb6145753f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:44.507068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6017041-9702-4a19-90f1-725ceb931821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:44.508507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6017041-9702-4a19-90f1-725ceb931821 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:44.530988 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1069/2131] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:44 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:27:44.654218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0029d17-7522-4dee-82a9-abdb6145753f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:44.663228 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11e1b8-ccab-4504-9d76-185b369f518f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac60b4d621234225830c94edfac400e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:44.664083 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11e1b8-ccab-4504-9d76-185b369f518f service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:44.671107 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11e1b8-ccab-4504-9d76-185b369f518f service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:44.671107 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11e1b8-ccab-4504-9d76-185b369f518f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:44.683772 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11e1b8-ccab-4504-9d76-185b369f518f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:44.685393 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11e1b8-ccab-4504-9d76-185b369f518f service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:44.695339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1063/2132] 10.222.0.39 () {60 vars in 1384 bytes} [Mon Aug 8 02:27:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:27:45.583689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6017041-9702-4a19-90f1-725ceb931821 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:45.699733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022dee0f-1a63-4c08-8894-33d3c788220c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:45.700258 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022dee0f-1a63-4c08-8894-33d3c788220c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:45.700423 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022dee0f-1a63-4c08-8894-33d3c788220c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:45.717504 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6017041-9702-4a19-90f1-725ceb931821 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:45.729917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb75ab0-3b24-4e99-9fe7-673ee6fab39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:45.731558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb75ab0-3b24-4e99-9fe7-673ee6fab39e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:45.731689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-022dee0f-1a63-4c08-8894-33d3c788220c None None] MFA Rules not processed for user `b88ccd3e5c4c48d3a27f4eaa7874e28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:45.731788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb75ab0-3b24-4e99-9fe7-673ee6fab39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:45.731788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb75ab0-3b24-4e99-9fe7-673ee6fab39e admin admin] PATH_INFO: `/v3/users/9563e12eb09e4970b47075837a63128b` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:45.732995 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-022dee0f-1a63-4c08-8894-33d3c788220c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:45.736964 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb75ab0-3b24-4e99-9fe7-673ee6fab39e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9563e12eb09e4970b47075837a63128b)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:45.739003 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb75ab0-3b24-4e99-9fe7-673ee6fab39e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:45.748244 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1064/2133] 10.222.0.39 () {58 vars in 967 bytes} [Mon Aug 8 02:27:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:27:45.765887 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022dee0f-1a63-4c08-8894-33d3c788220c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:45.789794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022dee0f-1a63-4c08-8894-33d3c788220c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:45.807250 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc6f5c-7a9e-45ec-977e-b704c7d1ba5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac60b4d621234225830c94edfac400e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:45.808919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc6f5c-7a9e-45ec-977e-b704c7d1ba5a service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:45.809551 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc6f5c-7a9e-45ec-977e-b704c7d1ba5a service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:45.811484 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc6f5c-7a9e-45ec-977e-b704c7d1ba5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:45.831979 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc6f5c-7a9e-45ec-977e-b704c7d1ba5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:45.833906 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc6f5c-7a9e-45ec-977e-b704c7d1ba5a service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:45.843691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9cb75ab0-3b24-4e99-9fe7-673ee6fab39e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9563e12eb09e4970b47075837a63128b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:45.844692 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1065/2134] 10.222.0.39 () {60 vars in 1384 bytes} [Mon Aug 8 02:27:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 02:27:45.853366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9cb75ab0-3b24-4e99-9fe7-673ee6fab39e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9563e12eb09e4970b47075837a63128b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:45.867467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9cb75ab0-3b24-4e99-9fe7-673ee6fab39e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9563e12eb09e4970b47075837a63128b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:45.868820 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9cb75ab0-3b24-4e99-9fe7-673ee6fab39e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9563e12eb09e4970b47075837a63128b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:45.875108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9cb75ab0-3b24-4e99-9fe7-673ee6fab39e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9563e12eb09e4970b47075837a63128b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:45.915588 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1070/2135] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:45 2022] DELETE /identity/v3/users/9563e12eb09e4970b47075837a63128b => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:45.922276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaafad2-fd57-4ec0-a50b-5fe4b83b85ba None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:45.922828 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaafad2-fd57-4ec0-a50b-5fe4b83b85ba None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:45.923835 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaafad2-fd57-4ec0-a50b-5fe4b83b85ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:45.974335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0eaafad2-fd57-4ec0-a50b-5fe4b83b85ba None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:45.975891 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0eaafad2-fd57-4ec0-a50b-5fe4b83b85ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:46.098383 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1066/2136] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:46.117255 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb75ab0-3b24-4e99-9fe7-673ee6fab39e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:46.119239 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9cb75ab0-3b24-4e99-9fe7-673ee6fab39e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:46.204498 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb75ab0-3b24-4e99-9fe7-673ee6fab39e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:46.222451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09a629-54e3-4dd1-bdeb-d1ab21beae2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:46.223121 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09a629-54e3-4dd1-bdeb-d1ab21beae2d service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:46.225787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09a629-54e3-4dd1-bdeb-d1ab21beae2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:46.226231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09a629-54e3-4dd1-bdeb-d1ab21beae2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:46.245512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b09a629-54e3-4dd1-bdeb-d1ab21beae2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:46.248273 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b09a629-54e3-4dd1-bdeb-d1ab21beae2d service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:46.261402 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f81495-ede7-43ae-bbf5-f07e0e9dd373 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:46.262086 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f81495-ede7-43ae-bbf5-f07e0e9dd373 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:46.262539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f81495-ede7-43ae-bbf5-f07e0e9dd373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:46.270118 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1071/2137] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:46.271145 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09a629-54e3-4dd1-bdeb-d1ab21beae2d service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:46.272508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2b09a629-54e3-4dd1-bdeb-d1ab21beae2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:46.274800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:46.289726 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c3f81495-ede7-43ae-bbf5-f07e0e9dd373 None None] MFA Rules not processed for user `b88ccd3e5c4c48d3a27f4eaa7874e28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:46.289891 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09a629-54e3-4dd1-bdeb-d1ab21beae2d service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:46.291306 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c3f81495-ede7-43ae-bbf5-f07e0e9dd373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:46.294588 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30586da6-2fb1-4a97-a69c-7a3d9ecd0f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:46.295214 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30586da6-2fb1-4a97-a69c-7a3d9ecd0f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:46.295497 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30586da6-2fb1-4a97-a69c-7a3d9ecd0f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:46.295782 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30586da6-2fb1-4a97-a69c-7a3d9ecd0f44 admin admin] PATH_INFO: `/v3/users/3ee31a15cc0a481a85a8d0e161217df4` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:46.297557 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30586da6-2fb1-4a97-a69c-7a3d9ecd0f44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ee31a15cc0a481a85a8d0e161217df4)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:46.299522 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30586da6-2fb1-4a97-a69c-7a3d9ecd0f44 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:46.319865 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:46.364609 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1067/2138] 10.222.0.39 () {58 vars in 1009 bytes} [Mon Aug 8 02:27:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:46.366908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-30586da6-2fb1-4a97-a69c-7a3d9ecd0f44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ee31a15cc0a481a85a8d0e161217df4'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:46.374983 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-30586da6-2fb1-4a97-a69c-7a3d9ecd0f44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ee31a15cc0a481a85a8d0e161217df4'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:46.379732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f81495-ede7-43ae-bbf5-f07e0e9dd373 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:46.381298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c3f81495-ede7-43ae-bbf5-f07e0e9dd373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:46.392131 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-30586da6-2fb1-4a97-a69c-7a3d9ecd0f44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ee31a15cc0a481a85a8d0e161217df4'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:46.392875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-30586da6-2fb1-4a97-a69c-7a3d9ecd0f44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ee31a15cc0a481a85a8d0e161217df4'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:46.393613 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-30586da6-2fb1-4a97-a69c-7a3d9ecd0f44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ee31a15cc0a481a85a8d0e161217df4'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:46.426577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1072/2139] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:46 2022] DELETE /identity/v3/users/3ee31a15cc0a481a85a8d0e161217df4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:46.513371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f81495-ede7-43ae-bbf5-f07e0e9dd373 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:46.605010 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ab97f-d6c3-44cf-827b-17be3e243c50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac60b4d621234225830c94edfac400e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:46.605622 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ab97f-d6c3-44cf-827b-17be3e243c50 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:46.605901 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ab97f-d6c3-44cf-827b-17be3e243c50 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:46.606171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ab97f-d6c3-44cf-827b-17be3e243c50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:46.608183 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-698ab97f-d6c3-44cf-827b-17be3e243c50 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:46.648137 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30586da6-2fb1-4a97-a69c-7a3d9ecd0f44 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:46.649695 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-30586da6-2fb1-4a97-a69c-7a3d9ecd0f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:46.680111 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:46.685338 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ab97f-d6c3-44cf-827b-17be3e243c50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:46.687689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ab97f-d6c3-44cf-827b-17be3e243c50 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:46.700314 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1068/2140] 10.222.0.39 () {60 vars in 1384 bytes} [Mon Aug 8 02:27:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 02:27:46.730972 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30586da6-2fb1-4a97-a69c-7a3d9ecd0f44 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:46.736070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97495504-c396-4046-a187-511ada20afd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:46.736785 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97495504-c396-4046-a187-511ada20afd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:46.737211 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97495504-c396-4046-a187-511ada20afd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:46.737608 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97495504-c396-4046-a187-511ada20afd9 admin admin] PATH_INFO: `/v3/projects/e84c6c5ce3cd45f48c581553d10fdc10` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:46.739254 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97495504-c396-4046-a187-511ada20afd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e84c6c5ce3cd45f48c581553d10fdc10)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:46.741437 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97495504-c396-4046-a187-511ada20afd9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:46.882584 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-97495504-c396-4046-a187-511ada20afd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e84c6c5ce3cd45f48c581553d10fdc10', 'request_id': 'req-97495504-c396-4046-a187-511ada20afd9'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:46.911814 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-97495504-c396-4046-a187-511ada20afd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e84c6c5ce3cd45f48c581553d10fdc10', 'request_id': 'req-97495504-c396-4046-a187-511ada20afd9'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:46.924957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-97495504-c396-4046-a187-511ada20afd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e84c6c5ce3cd45f48c581553d10fdc10', 'request_id': 'req-97495504-c396-4046-a187-511ada20afd9'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:46.963147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-97495504-c396-4046-a187-511ada20afd9 admin admin] The token cache is being invalidate because project e84c6c5ce3cd45f48c581553d10fdc10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:46.964255 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-97495504-c396-4046-a187-511ada20afd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:46.964722 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ab97f-d6c3-44cf-827b-17be3e243c50 service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:46.966039 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-97495504-c396-4046-a187-511ada20afd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:46.966467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-698ab97f-d6c3-44cf-827b-17be3e243c50 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:46.968935 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1073/2141] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:46 2022] DELETE /identity/v3/projects/e84c6c5ce3cd45f48c581553d10fdc10 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:27:46.974441 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488350b0-780a-4156-8bca-28f79d888809 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:46.974619 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488350b0-780a-4156-8bca-28f79d888809 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:46.975555 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488350b0-780a-4156-8bca-28f79d888809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:46.997462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ab97f-d6c3-44cf-827b-17be3e243c50 service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:47.043406 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-488350b0-780a-4156-8bca-28f79d888809 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:47.045010 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-488350b0-780a-4156-8bca-28f79d888809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:47.063379 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:47.084750 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51c66b-9872-4929-90b4-678b01355bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:47.085459 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51c66b-9872-4929-90b4-678b01355bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:47.086003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51c66b-9872-4929-90b4-678b01355bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:47.086494 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51c66b-9872-4929-90b4-678b01355bca admin admin] PATH_INFO: `/v3/projects/7e8274dd2d5b4d028e805d59f4ac8a75` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:47.088132 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51c66b-9872-4929-90b4-678b01355bca admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e8274dd2d5b4d028e805d59f4ac8a75)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:47.092771 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51c66b-9872-4929-90b4-678b01355bca admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:47.154624 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1074/2142] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:47.156039 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c49cf-51aa-48cc-b745-f3aecaa9a7c5 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:47.156500 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c49cf-51aa-48cc-b745-f3aecaa9a7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:47.156766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c49cf-51aa-48cc-b745-f3aecaa9a7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:47.170562 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bf51c66b-9872-4929-90b4-678b01355bca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e8274dd2d5b4d028e805d59f4ac8a75', 'request_id': 'req-bf51c66b-9872-4929-90b4-678b01355bca'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:47.183529 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bf51c66b-9872-4929-90b4-678b01355bca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e8274dd2d5b4d028e805d59f4ac8a75', 'request_id': 'req-bf51c66b-9872-4929-90b4-678b01355bca'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:47.191569 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-015c49cf-51aa-48cc-b745-f3aecaa9a7c5 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:47.193201 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-015c49cf-51aa-48cc-b745-f3aecaa9a7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:47.207936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bf51c66b-9872-4929-90b4-678b01355bca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e8274dd2d5b4d028e805d59f4ac8a75', 'request_id': 'req-bf51c66b-9872-4929-90b4-678b01355bca'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:47.224608 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bf51c66b-9872-4929-90b4-678b01355bca admin admin] The token cache is being invalidate because project 7e8274dd2d5b4d028e805d59f4ac8a75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:47.224965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bf51c66b-9872-4929-90b4-678b01355bca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:47.225791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bf51c66b-9872-4929-90b4-678b01355bca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:47.230134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1069/2143] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:46 2022] DELETE /identity/v3/projects/7e8274dd2d5b4d028e805d59f4ac8a75 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:27:47.230134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc882b-589d-4676-a976-104572212af8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:47.230134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc882b-589d-4676-a976-104572212af8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:47.230134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc882b-589d-4676-a976-104572212af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:47.256069 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-31cc882b-589d-4676-a976-104572212af8 None None] MFA Rules not processed for user `b88ccd3e5c4c48d3a27f4eaa7874e28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:47.257293 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-31cc882b-589d-4676-a976-104572212af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:47.314908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1075/2144] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:47.320928 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c49cf-51aa-48cc-b745-f3aecaa9a7c5 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:47.322397 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-015c49cf-51aa-48cc-b745-f3aecaa9a7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:47.334509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1070/2145] 10.222.0.39 () {58 vars in 967 bytes} [Mon Aug 8 02:27:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:47.335593 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc882b-589d-4676-a976-104572212af8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:47.337234 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c49cf-51aa-48cc-b745-f3aecaa9a7c5 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:47.344126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a097c21-ae14-42ea-b522-97fd5db0ac31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:47.344928 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a097c21-ae14-42ea-b522-97fd5db0ac31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:47.345199 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a097c21-ae14-42ea-b522-97fd5db0ac31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:47.345751 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a097c21-ae14-42ea-b522-97fd5db0ac31 admin admin] PATH_INFO: `/v3/users/1f5d9026aa284cbe9a23fd3c30e6361f` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:47.347081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a097c21-ae14-42ea-b522-97fd5db0ac31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f5d9026aa284cbe9a23fd3c30e6361f)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:47.349318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a097c21-ae14-42ea-b522-97fd5db0ac31 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:47.352802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc882b-589d-4676-a976-104572212af8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:47.358447 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517641ab-a43e-414f-80c2-5c87e0250830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:47.359192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517641ab-a43e-414f-80c2-5c87e0250830 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:47.359576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517641ab-a43e-414f-80c2-5c87e0250830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:47.359916 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517641ab-a43e-414f-80c2-5c87e0250830 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:47.360675 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517641ab-a43e-414f-80c2-5c87e0250830 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:27:47.361041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517641ab-a43e-414f-80c2-5c87e0250830 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:47.362765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517641ab-a43e-414f-80c2-5c87e0250830 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:47.376275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1071/2146] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:27:47.377457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517641ab-a43e-414f-80c2-5c87e0250830 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:47.383314 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-517641ab-a43e-414f-80c2-5c87e0250830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:47.409049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:47.425028 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9a097c21-ae14-42ea-b522-97fd5db0ac31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f5d9026aa284cbe9a23fd3c30e6361f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:47.434078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9a097c21-ae14-42ea-b522-97fd5db0ac31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f5d9026aa284cbe9a23fd3c30e6361f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:47.447149 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9a097c21-ae14-42ea-b522-97fd5db0ac31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f5d9026aa284cbe9a23fd3c30e6361f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:47.448829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9a097c21-ae14-42ea-b522-97fd5db0ac31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f5d9026aa284cbe9a23fd3c30e6361f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:47.449861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9a097c21-ae14-42ea-b522-97fd5db0ac31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f5d9026aa284cbe9a23fd3c30e6361f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:47.473406 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1076/2147] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:47 2022] DELETE /identity/v3/users/1f5d9026aa284cbe9a23fd3c30e6361f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:47.474391 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a097c21-ae14-42ea-b522-97fd5db0ac31 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:47.475003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517641ab-a43e-414f-80c2-5c87e0250830 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:47.479063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9a097c21-ae14-42ea-b522-97fd5db0ac31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:47.527611 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:47.560602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0f81c-f7ad-401a-9310-d6e56b55e600 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac60b4d621234225830c94edfac400e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:47.561546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0f81c-f7ad-401a-9310-d6e56b55e600 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:47.561841 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0f81c-f7ad-401a-9310-d6e56b55e600 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:47.562120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0f81c-f7ad-401a-9310-d6e56b55e600 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:47.565817 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-add0f81c-f7ad-401a-9310-d6e56b55e600 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:47.572292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a097c21-ae14-42ea-b522-97fd5db0ac31 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:47.577675 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99355e46-53a1-4d21-9599-b5fcb894fd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:47.578464 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99355e46-53a1-4d21-9599-b5fcb894fd4e admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:47.578960 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99355e46-53a1-4d21-9599-b5fcb894fd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:47.579306 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99355e46-53a1-4d21-9599-b5fcb894fd4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:47.580016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99355e46-53a1-4d21-9599-b5fcb894fd4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:47.582127 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99355e46-53a1-4d21-9599-b5fcb894fd4e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:47.612301 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1077/2148] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:47 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:47.617158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99355e46-53a1-4d21-9599-b5fcb894fd4e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:47.653719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0f81c-f7ad-401a-9310-d6e56b55e600 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:47.655875 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0f81c-f7ad-401a-9310-d6e56b55e600 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:47.665562 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1072/2149] 10.222.0.39 () {60 vars in 1384 bytes} [Mon Aug 8 02:27:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:27:47.692249 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99355e46-53a1-4d21-9599-b5fcb894fd4e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:47.699810 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424692d-dfa9-4983-a6a9-0938708ad454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:47.700506 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424692d-dfa9-4983-a6a9-0938708ad454 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:47.700855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424692d-dfa9-4983-a6a9-0938708ad454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:47.701197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424692d-dfa9-4983-a6a9-0938708ad454 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:47.702029 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424692d-dfa9-4983-a6a9-0938708ad454 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:47.704125 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424692d-dfa9-4983-a6a9-0938708ad454 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:47.733724 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1078/2150] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:47 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:47.737812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0f81c-f7ad-401a-9310-d6e56b55e600 service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:47.750795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0f81c-f7ad-401a-9310-d6e56b55e600 service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:47.756916 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edced0ae-780d-47c8-b9b5-99957ad5bee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:47.757744 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edced0ae-780d-47c8-b9b5-99957ad5bee7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:47.757927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edced0ae-780d-47c8-b9b5-99957ad5bee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:47.758408 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edced0ae-780d-47c8-b9b5-99957ad5bee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:47.758998 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edced0ae-780d-47c8-b9b5-99957ad5bee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:47.761648 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edced0ae-780d-47c8-b9b5-99957ad5bee7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:47.774324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1073/2151] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:47.778894 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424692d-dfa9-4983-a6a9-0938708ad454 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:47.790768 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424692d-dfa9-4983-a6a9-0938708ad454 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:47.799640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b67c6-5aa3-461e-836e-842529cab819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:47.799855 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edced0ae-780d-47c8-b9b5-99957ad5bee7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:47.800804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b67c6-5aa3-461e-836e-842529cab819 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:47.801098 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b67c6-5aa3-461e-836e-842529cab819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:47.801442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b67c6-5aa3-461e-836e-842529cab819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:47.802235 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b67c6-5aa3-461e-836e-842529cab819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:47.804360 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-edced0ae-780d-47c8-b9b5-99957ad5bee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:47.804499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b67c6-5aa3-461e-836e-842529cab819 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:47.820421 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edced0ae-780d-47c8-b9b5-99957ad5bee7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:47.829396 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1079/2152] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:27:47.835175 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc3232-2e48-476d-9cee-258733f39dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:47.835781 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b67c6-5aa3-461e-836e-842529cab819 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:47.837449 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc3232-2e48-476d-9cee-258733f39dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:47.837936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc3232-2e48-476d-9cee-258733f39dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:47.846649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc3232-2e48-476d-9cee-258733f39dbe admin admin] PATH_INFO: `/v3/projects/9bac15c2479547768ced270aef0d782d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:47.848076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b67c6-5aa3-461e-836e-842529cab819 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:47.849398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc3232-2e48-476d-9cee-258733f39dbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bac15c2479547768ced270aef0d782d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:47.851486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc3232-2e48-476d-9cee-258733f39dbe admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:47.855400 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d5c9e-26f4-42be-bac8-6be2e3693fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:47.857166 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d5c9e-26f4-42be-bac8-6be2e3693fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:47.857671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d5c9e-26f4-42be-bac8-6be2e3693fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:47.857951 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d5c9e-26f4-42be-bac8-6be2e3693fdd admin admin] PATH_INFO: `/v3/projects/066baf4af3be4dbc89e510a401ec573b/users/af41e4b9556d45209108812c043744ca/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:47.878953 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d5c9e-26f4-42be-bac8-6be2e3693fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=066baf4af3be4dbc89e510a401ec573b, user_id=af41e4b9556d45209108812c043744ca, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:47.881721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d5c9e-26f4-42be-bac8-6be2e3693fdd admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:47.910288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1080/2153] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:47 2022] PUT /identity/v3/projects/066baf4af3be4dbc89e510a401ec573b/users/af41e4b9556d45209108812c043744ca/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:47.921867 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f1733-382d-42f2-a8d4-16d086de3484 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:47.922152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f1733-382d-42f2-a8d4-16d086de3484 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:47.922869 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f1733-382d-42f2-a8d4-16d086de3484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:47.935414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:47.945082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-72cc3232-2e48-476d-9cee-258733f39dbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bac15c2479547768ced270aef0d782d', 'request_id': 'req-72cc3232-2e48-476d-9cee-258733f39dbe'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:47.953882 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-72cc3232-2e48-476d-9cee-258733f39dbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bac15c2479547768ced270aef0d782d', 'request_id': 'req-72cc3232-2e48-476d-9cee-258733f39dbe'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:47.958154 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ae0f1733-382d-42f2-a8d4-16d086de3484 None None] MFA Rules not processed for user `af41e4b9556d45209108812c043744ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:47.964943 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-72cc3232-2e48-476d-9cee-258733f39dbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bac15c2479547768ced270aef0d782d', 'request_id': 'req-72cc3232-2e48-476d-9cee-258733f39dbe'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:47.974629 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ae0f1733-382d-42f2-a8d4-16d086de3484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:47.981297 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-72cc3232-2e48-476d-9cee-258733f39dbe admin admin] The token cache is being invalidate because project 9bac15c2479547768ced270aef0d782d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:47.981675 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-72cc3232-2e48-476d-9cee-258733f39dbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:47.982461 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-72cc3232-2e48-476d-9cee-258733f39dbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:47.984421 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1074/2154] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:47 2022] DELETE /identity/v3/projects/9bac15c2479547768ced270aef0d782d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:27:47.988918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d232ce4-c0b4-4360-b6b7-497780277e39 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:47.989199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d232ce4-c0b4-4360-b6b7-497780277e39 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:47.989472 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d232ce4-c0b4-4360-b6b7-497780277e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:48.013799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5d232ce4-c0b4-4360-b6b7-497780277e39 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:48.015234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5d232ce4-c0b4-4360-b6b7-497780277e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:48.075301 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:48.087569 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1081/2155] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:27:48.088639 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f1733-382d-42f2-a8d4-16d086de3484 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:48.089992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ae0f1733-382d-42f2-a8d4-16d086de3484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:48.118431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1075/2156] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:48.126170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef7ecf-f7b2-4366-9664-06c78bbc8b66 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:48.126505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef7ecf-f7b2-4366-9664-06c78bbc8b66 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:48.126778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef7ecf-f7b2-4366-9664-06c78bbc8b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:48.145728 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c6ef7ecf-f7b2-4366-9664-06c78bbc8b66 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:48.147000 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c6ef7ecf-f7b2-4366-9664-06c78bbc8b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:48.156245 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1076/2157] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:27:48.160240 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef7ecf-f7b2-4366-9664-06c78bbc8b66 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:48.167429 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f1733-382d-42f2-a8d4-16d086de3484 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:48.170314 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef7ecf-f7b2-4366-9664-06c78bbc8b66 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:48.173025 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fe5dd-9213-49aa-9dc0-22dceca840bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:48.173970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fe5dd-9213-49aa-9dc0-22dceca840bd admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:48.174395 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fe5dd-9213-49aa-9dc0-22dceca840bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:48.174928 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fe5dd-9213-49aa-9dc0-22dceca840bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:48.175618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fe5dd-9213-49aa-9dc0-22dceca840bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:48.177724 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fe5dd-9213-49aa-9dc0-22dceca840bd admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:48.178167 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581da397-d1b7-407f-9b33-ebbbf3668ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:48.178976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581da397-d1b7-407f-9b33-ebbbf3668ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:48.179344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581da397-d1b7-407f-9b33-ebbbf3668ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:48.179693 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581da397-d1b7-407f-9b33-ebbbf3668ca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:48.180320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581da397-d1b7-407f-9b33-ebbbf3668ca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:27:48.180817 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581da397-d1b7-407f-9b33-ebbbf3668ca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:48.183267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581da397-d1b7-407f-9b33-ebbbf3668ca7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:48.195778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1077/2158] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:27:48.199817 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581da397-d1b7-407f-9b33-ebbbf3668ca7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:48.210091 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581da397-d1b7-407f-9b33-ebbbf3668ca7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:48.215882 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b89ec5-a089-4b86-9a35-91010cf5aa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:48.216626 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b89ec5-a089-4b86-9a35-91010cf5aa70 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:48.217039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b89ec5-a089-4b86-9a35-91010cf5aa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:48.217416 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b89ec5-a089-4b86-9a35-91010cf5aa70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:48.218139 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b89ec5-a089-4b86-9a35-91010cf5aa70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:48.220252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b89ec5-a089-4b86-9a35-91010cf5aa70 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:48.224338 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1082/2159] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:48 2022] POST /identity/v3/projects => generated 381 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:48.225227 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fe5dd-9213-49aa-9dc0-22dceca840bd admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:48.271737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1078/2160] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:48 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:27:48.271991 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b89ec5-a089-4b86-9a35-91010cf5aa70 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:48.328300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fe5dd-9213-49aa-9dc0-22dceca840bd admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:48.365135 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b89ec5-a089-4b86-9a35-91010cf5aa70 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:48.372669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1843c9f1-cd07-487d-ad75-ead65fc233a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:48.373691 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843c9f1-cd07-487d-ad75-ead65fc233a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:48.373691 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843c9f1-cd07-487d-ad75-ead65fc233a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:48.373911 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843c9f1-cd07-487d-ad75-ead65fc233a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:48.374450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1843c9f1-cd07-487d-ad75-ead65fc233a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:48.376772 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1843c9f1-cd07-487d-ad75-ead65fc233a3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:48.418359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1079/2161] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:48 2022] POST /identity/v3/users => generated 291 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:27:48.426790 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1843c9f1-cd07-487d-ad75-ead65fc233a3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:48.427894 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53317e95-ade2-409f-9e84-639d0f5d62f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:48.428521 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53317e95-ade2-409f-9e84-639d0f5d62f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:48.429525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53317e95-ade2-409f-9e84-639d0f5d62f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:48.434069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53317e95-ade2-409f-9e84-639d0f5d62f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:48.434764 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53317e95-ade2-409f-9e84-639d0f5d62f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:48.436809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53317e95-ade2-409f-9e84-639d0f5d62f1 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:48.444857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1843c9f1-cd07-487d-ad75-ead65fc233a3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:48.453168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7249d4a-6f53-453e-b084-d3830507c8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:48.453947 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7249d4a-6f53-453e-b084-d3830507c8ef admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:48.454226 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7249d4a-6f53-453e-b084-d3830507c8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:48.454545 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7249d4a-6f53-453e-b084-d3830507c8ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:48.455262 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7249d4a-6f53-453e-b084-d3830507c8ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:48.457218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7249d4a-6f53-453e-b084-d3830507c8ef admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:48.465453 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1083/2162] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:48 2022] POST /identity/v3/users => generated 297 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:48.467276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53317e95-ade2-409f-9e84-639d0f5d62f1 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:48.469550 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1080/2163] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:27:48.473899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7249d4a-6f53-453e-b084-d3830507c8ef admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:48.480755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53317e95-ade2-409f-9e84-639d0f5d62f1 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:48.486592 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f0f29-7cb0-4379-ae1c-11f9e7561007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:48.486730 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7249d4a-6f53-453e-b084-d3830507c8ef admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:48.487379 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f0f29-7cb0-4379-ae1c-11f9e7561007 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:48.487685 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f0f29-7cb0-4379-ae1c-11f9e7561007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:48.488029 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f0f29-7cb0-4379-ae1c-11f9e7561007 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:48.488675 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f0f29-7cb0-4379-ae1c-11f9e7561007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:48.490438 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f0f29-7cb0-4379-ae1c-11f9e7561007 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:48.492562 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49144aa3-86e8-4974-9c55-317032580237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:48.493155 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49144aa3-86e8-4974-9c55-317032580237 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:48.493432 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49144aa3-86e8-4974-9c55-317032580237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:48.493699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49144aa3-86e8-4974-9c55-317032580237 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:48.494358 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49144aa3-86e8-4974-9c55-317032580237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:48.496076 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49144aa3-86e8-4974-9c55-317032580237 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:48.498673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:48.504479 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1084/2164] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:48.507451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f0f29-7cb0-4379-ae1c-11f9e7561007 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:48.509093 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1081/2165] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:27:48.513449 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49144aa3-86e8-4974-9c55-317032580237 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:48.523239 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f0f29-7cb0-4379-ae1c-11f9e7561007 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:48.527602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49144aa3-86e8-4974-9c55-317032580237 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:48.529460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33646ad-bd27-4747-8a7f-1e9f2ad57959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:48.530111 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33646ad-bd27-4747-8a7f-1e9f2ad57959 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:48.530430 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33646ad-bd27-4747-8a7f-1e9f2ad57959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:48.530707 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33646ad-bd27-4747-8a7f-1e9f2ad57959 admin admin] PATH_INFO: `/v3/projects/9dcdcb70aeea49f190dcfb60c5687942/users/77eb133292ad405bb9e1eb37c2eb2a48/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:48.532939 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0958611-fe13-4dbe-b059-7d41687cade4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:48.533624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0958611-fe13-4dbe-b059-7d41687cade4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:48.534206 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0958611-fe13-4dbe-b059-7d41687cade4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:48.534517 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0958611-fe13-4dbe-b059-7d41687cade4 admin admin] PATH_INFO: `/v3/projects/00b0a7a94f64457ab14a90ea1ab5f593/users/19a3b92e4cd148c29e61bf04191af812/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:48.540851 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:48.551275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0958611-fe13-4dbe-b059-7d41687cade4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00b0a7a94f64457ab14a90ea1ab5f593, user_id=19a3b92e4cd148c29e61bf04191af812, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:48.553468 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33646ad-bd27-4747-8a7f-1e9f2ad57959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dcdcb70aeea49f190dcfb60c5687942, user_id=77eb133292ad405bb9e1eb37c2eb2a48, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:48.554101 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0958611-fe13-4dbe-b059-7d41687cade4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:48.556437 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33646ad-bd27-4747-8a7f-1e9f2ad57959 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:48.572829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1085/2166] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:48 2022] PUT /identity/v3/projects/9dcdcb70aeea49f190dcfb60c5687942/users/77eb133292ad405bb9e1eb37c2eb2a48/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:48.575804 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1082/2167] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:48 2022] PUT /identity/v3/projects/00b0a7a94f64457ab14a90ea1ab5f593/users/19a3b92e4cd148c29e61bf04191af812/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:27:48.576754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33646ad-bd27-4747-8a7f-1e9f2ad57959 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:48.590743 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd23913-61d6-4a2c-bd8f-aa52a5efc7e5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:48.591049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd23913-61d6-4a2c-bd8f-aa52a5efc7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:48.591326 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd23913-61d6-4a2c-bd8f-aa52a5efc7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:48.621987 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8cd23913-61d6-4a2c-bd8f-aa52a5efc7e5 None None] MFA Rules not processed for user `19a3b92e4cd148c29e61bf04191af812`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:48.623266 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8cd23913-61d6-4a2c-bd8f-aa52a5efc7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:48.650220 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33646ad-bd27-4747-8a7f-1e9f2ad57959 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:48.698359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1083/2168] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:27:48.701742 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd23913-61d6-4a2c-bd8f-aa52a5efc7e5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:48.725493 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056979b-7a5d-4800-921d-7cb520a2fdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:48.726095 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056979b-7a5d-4800-921d-7cb520a2fdf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:48.726402 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056979b-7a5d-4800-921d-7cb520a2fdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:48.726676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056979b-7a5d-4800-921d-7cb520a2fdf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:48.727750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056979b-7a5d-4800-921d-7cb520a2fdf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:48.729547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056979b-7a5d-4800-921d-7cb520a2fdf2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:48.742681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1086/2169] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:48.747075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056979b-7a5d-4800-921d-7cb520a2fdf2 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:48.756970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056979b-7a5d-4800-921d-7cb520a2fdf2 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:48.763359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b30e6-d34c-42d5-99ba-36ad9cb610da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:48.764008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b30e6-d34c-42d5-99ba-36ad9cb610da admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:48.764281 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b30e6-d34c-42d5-99ba-36ad9cb610da admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:48.764556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b30e6-d34c-42d5-99ba-36ad9cb610da admin admin] PATH_INFO: `/v3/projects/9dcdcb70aeea49f190dcfb60c5687942/users/77eb133292ad405bb9e1eb37c2eb2a48/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:48.766620 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b30e6-d34c-42d5-99ba-36ad9cb610da admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dcdcb70aeea49f190dcfb60c5687942, user_id=77eb133292ad405bb9e1eb37c2eb2a48, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:48.769272 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b30e6-d34c-42d5-99ba-36ad9cb610da admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:48.782041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1087/2170] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:48 2022] PUT /identity/v3/projects/9dcdcb70aeea49f190dcfb60c5687942/users/77eb133292ad405bb9e1eb37c2eb2a48/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:48.789979 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd23913-61d6-4a2c-bd8f-aa52a5efc7e5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:48.792269 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e360-3142-46d0-8968-d73f6c823332 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:48.792407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e360-3142-46d0-8968-d73f6c823332 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:48.792685 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e360-3142-46d0-8968-d73f6c823332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:48.823745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f321e360-3142-46d0-8968-d73f6c823332 None None] MFA Rules not processed for user `77eb133292ad405bb9e1eb37c2eb2a48`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:48.824983 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f321e360-3142-46d0-8968-d73f6c823332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:48.860059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341caeb-9c4f-4c80-872e-345da036fcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:48.860663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341caeb-9c4f-4c80-872e-345da036fcd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:48.860947 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341caeb-9c4f-4c80-872e-345da036fcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:48.861239 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341caeb-9c4f-4c80-872e-345da036fcd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:48.861863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341caeb-9c4f-4c80-872e-345da036fcd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:48.863836 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341caeb-9c4f-4c80-872e-345da036fcd7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:48.880233 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:48.897129 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1084/2171] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:48 2022] POST /identity/v3/projects => generated 379 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:27:48.901628 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341caeb-9c4f-4c80-872e-345da036fcd7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:48.914233 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1088/2172] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:27:48.926379 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321e360-3142-46d0-8968-d73f6c823332 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:48.927814 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f321e360-3142-46d0-8968-d73f6c823332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:48.933069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:48.975019 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341caeb-9c4f-4c80-872e-345da036fcd7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:48.980249 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0640071-b3dd-4578-bf20-f6ec122354a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:48.980853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0640071-b3dd-4578-bf20-f6ec122354a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:48.981147 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0640071-b3dd-4578-bf20-f6ec122354a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:48.981422 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0640071-b3dd-4578-bf20-f6ec122354a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:48.982078 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0640071-b3dd-4578-bf20-f6ec122354a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:48.984064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0640071-b3dd-4578-bf20-f6ec122354a4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:48.993774 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321e360-3142-46d0-8968-d73f6c823332 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:49.005634 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1085/2173] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:48 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:27:49.009531 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0640071-b3dd-4578-bf20-f6ec122354a4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:49.019650 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0640071-b3dd-4578-bf20-f6ec122354a4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:49.024490 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b614a-8e31-4e70-b0f7-372d8ea5c137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:49.025109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b614a-8e31-4e70-b0f7-372d8ea5c137 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:49.025387 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b614a-8e31-4e70-b0f7-372d8ea5c137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:49.025664 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b614a-8e31-4e70-b0f7-372d8ea5c137 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:49.026892 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b614a-8e31-4e70-b0f7-372d8ea5c137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:49.028521 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b614a-8e31-4e70-b0f7-372d8ea5c137 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:49.040374 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1086/2174] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:27:49.044342 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b614a-8e31-4e70-b0f7-372d8ea5c137 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:49.051517 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e563f11-d9a0-424f-81e9-a1fc2b5ded5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:49.052360 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e563f11-d9a0-424f-81e9-a1fc2b5ded5b service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:49.052766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e563f11-d9a0-424f-81e9-a1fc2b5ded5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:49.053039 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e563f11-d9a0-424f-81e9-a1fc2b5ded5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:49.054015 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b614a-8e31-4e70-b0f7-372d8ea5c137 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:49.059179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bf02b-023b-4b7e-9bc5-4eb919546cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:49.059809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bf02b-023b-4b7e-9bc5-4eb919546cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:49.060090 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bf02b-023b-4b7e-9bc5-4eb919546cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:49.060363 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bf02b-023b-4b7e-9bc5-4eb919546cdd admin admin] PATH_INFO: `/v3/projects/34464e76b222459396c9dce08646e224/users/65623a8aefc747d98cf63d109c990fe0/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:49.078485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829bf02b-023b-4b7e-9bc5-4eb919546cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=34464e76b222459396c9dce08646e224, user_id=65623a8aefc747d98cf63d109c990fe0, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:49.082403 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829bf02b-023b-4b7e-9bc5-4eb919546cdd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:49.108523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1087/2175] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:49 2022] PUT /identity/v3/projects/34464e76b222459396c9dce08646e224/users/65623a8aefc747d98cf63d109c990fe0/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:27:49.112720 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bf02b-023b-4b7e-9bc5-4eb919546cdd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:49.120918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e563f11-d9a0-424f-81e9-a1fc2b5ded5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:49.122480 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e563f11-d9a0-424f-81e9-a1fc2b5ded5b service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:49.132706 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1089/2176] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:48 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:27:49.188820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bf02b-023b-4b7e-9bc5-4eb919546cdd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:49.196440 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0527a9-4259-4095-bc30-a3046bfb6d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:49.197141 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0527a9-4259-4095-bc30-a3046bfb6d80 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:49.197441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0527a9-4259-4095-bc30-a3046bfb6d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:49.198494 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0527a9-4259-4095-bc30-a3046bfb6d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:49.199174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0527a9-4259-4095-bc30-a3046bfb6d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:49.200916 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0527a9-4259-4095-bc30-a3046bfb6d80 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:49.218167 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1088/2177] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:27:49.222896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e563f11-d9a0-424f-81e9-a1fc2b5ded5b service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:49.237786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0527a9-4259-4095-bc30-a3046bfb6d80 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:49.237976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e563f11-d9a0-424f-81e9-a1fc2b5ded5b service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:49.243661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99518739-832c-4077-8de2-6d41d3576996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:49.244324 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99518739-832c-4077-8de2-6d41d3576996 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:49.245213 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99518739-832c-4077-8de2-6d41d3576996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:49.245514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99518739-832c-4077-8de2-6d41d3576996 admin admin] PATH_INFO: `/v3/projects/34464e76b222459396c9dce08646e224/users/65623a8aefc747d98cf63d109c990fe0/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:49.251816 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99518739-832c-4077-8de2-6d41d3576996 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34464e76b222459396c9dce08646e224, user_id=65623a8aefc747d98cf63d109c990fe0, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:49.255059 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99518739-832c-4077-8de2-6d41d3576996 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:49.268299 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1090/2178] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:49 2022] PUT /identity/v3/projects/34464e76b222459396c9dce08646e224/users/65623a8aefc747d98cf63d109c990fe0/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:27:49.279658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3f9d9-8a72-4e0e-b552-ee4238f88426 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:49.279938 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3f9d9-8a72-4e0e-b552-ee4238f88426 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:49.280204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3f9d9-8a72-4e0e-b552-ee4238f88426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:49.308781 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-dbf3f9d9-8a72-4e0e-b552-ee4238f88426 None None] MFA Rules not processed for user `65623a8aefc747d98cf63d109c990fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:49.310111 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dbf3f9d9-8a72-4e0e-b552-ee4238f88426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:49.310733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:49.317913 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0527a9-4259-4095-bc30-a3046bfb6d80 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:49.356719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:49.368002 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9335c31-efba-4739-bdc1-362f34113f84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:49.368608 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9335c31-efba-4739-bdc1-362f34113f84 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:49.368915 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9335c31-efba-4739-bdc1-362f34113f84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:49.369189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9335c31-efba-4739-bdc1-362f34113f84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:49.383295 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1091/2179] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:49 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:27:49.394850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3f9d9-8a72-4e0e-b552-ee4238f88426 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:49.396230 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dbf3f9d9-8a72-4e0e-b552-ee4238f88426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:49.408715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3f9d9-8a72-4e0e-b552-ee4238f88426 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:49.412400 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a00a23-18e8-4dda-b7a1-c25f1a062790 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:49.412985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a00a23-18e8-4dda-b7a1-c25f1a062790 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:49.413263 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a00a23-18e8-4dda-b7a1-c25f1a062790 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:49.413531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a00a23-18e8-4dda-b7a1-c25f1a062790 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:49.427355 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a00a23-18e8-4dda-b7a1-c25f1a062790 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:49.427616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9335c31-efba-4739-bdc1-362f34113f84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:49.428822 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a00a23-18e8-4dda-b7a1-c25f1a062790 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:49.429056 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9335c31-efba-4739-bdc1-362f34113f84 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:49.438621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1089/2180] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:27:49.446440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1092/2181] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:49 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:27:49.745302 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9335c31-efba-4739-bdc1-362f34113f84 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:49.815583 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9335c31-efba-4739-bdc1-362f34113f84 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:49.821926 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3929471-c99f-4c32-8e5d-c7f02f5d4722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:49.822711 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3929471-c99f-4c32-8e5d-c7f02f5d4722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:49.823088 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3929471-c99f-4c32-8e5d-c7f02f5d4722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:49.823434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3929471-c99f-4c32-8e5d-c7f02f5d4722 admin admin] PATH_INFO: `/v3/users/af41e4b9556d45209108812c043744ca` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:49.824867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3929471-c99f-4c32-8e5d-c7f02f5d4722 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af41e4b9556d45209108812c043744ca)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:49.826923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3929471-c99f-4c32-8e5d-c7f02f5d4722 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:49.858832 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a00a23-18e8-4dda-b7a1-c25f1a062790 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:49.870036 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a00a23-18e8-4dda-b7a1-c25f1a062790 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:49.874354 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745a6fd-6170-465c-9f3e-ded1348b1287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:49.874970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745a6fd-6170-465c-9f3e-ded1348b1287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:49.875250 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745a6fd-6170-465c-9f3e-ded1348b1287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:49.875524 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745a6fd-6170-465c-9f3e-ded1348b1287 admin admin] PATH_INFO: `/v3/users/19a3b92e4cd148c29e61bf04191af812` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:49.876647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745a6fd-6170-465c-9f3e-ded1348b1287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19a3b92e4cd148c29e61bf04191af812)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:49.878589 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745a6fd-6170-465c-9f3e-ded1348b1287 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:49.882595 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c3929471-c99f-4c32-8e5d-c7f02f5d4722 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af41e4b9556d45209108812c043744ca'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:49.891198 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c3929471-c99f-4c32-8e5d-c7f02f5d4722 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af41e4b9556d45209108812c043744ca'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:49.895770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:49.899053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c3929471-c99f-4c32-8e5d-c7f02f5d4722 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af41e4b9556d45209108812c043744ca'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:49.900028 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c3929471-c99f-4c32-8e5d-c7f02f5d4722 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af41e4b9556d45209108812c043744ca'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:49.901267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c3929471-c99f-4c32-8e5d-c7f02f5d4722 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af41e4b9556d45209108812c043744ca'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:49.922552 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1090/2182] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:49 2022] DELETE /identity/v3/users/af41e4b9556d45209108812c043744ca => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:49.926891 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbea2bd-a3ef-4f1d-965a-fed805b3e6f7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:49.927167 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbea2bd-a3ef-4f1d-965a-fed805b3e6f7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:49.927416 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8745a6fd-6170-465c-9f3e-ded1348b1287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19a3b92e4cd148c29e61bf04191af812'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:49.927523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbea2bd-a3ef-4f1d-965a-fed805b3e6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:49.934722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8745a6fd-6170-465c-9f3e-ded1348b1287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19a3b92e4cd148c29e61bf04191af812'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:49.943187 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8745a6fd-6170-465c-9f3e-ded1348b1287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19a3b92e4cd148c29e61bf04191af812'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:49.943848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8745a6fd-6170-465c-9f3e-ded1348b1287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19a3b92e4cd148c29e61bf04191af812'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:49.944480 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8745a6fd-6170-465c-9f3e-ded1348b1287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19a3b92e4cd148c29e61bf04191af812'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:49.947507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7fbea2bd-a3ef-4f1d-965a-fed805b3e6f7 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:49.948758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7fbea2bd-a3ef-4f1d-965a-fed805b3e6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:49.968610 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1093/2183] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:49 2022] DELETE /identity/v3/users/19a3b92e4cd148c29e61bf04191af812 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:27:49.969934 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3293920-3968-42a8-b162-20ac8db25641 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:49.970265 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3293920-3968-42a8-b162-20ac8db25641 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:49.970534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3293920-3968-42a8-b162-20ac8db25641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:49.991178 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a3293920-3968-42a8-b162-20ac8db25641 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:49.991178 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a3293920-3968-42a8-b162-20ac8db25641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:50.011534 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1091/2184] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:27:50.019133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbea2bd-a3ef-4f1d-965a-fed805b3e6f7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:50.020642 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7fbea2bd-a3ef-4f1d-965a-fed805b3e6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:50.033431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbea2bd-a3ef-4f1d-965a-fed805b3e6f7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:50.061748 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1094/2185] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:50.071290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3293920-3968-42a8-b162-20ac8db25641 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:50.073603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a3293920-3968-42a8-b162-20ac8db25641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:50.100792 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8553a0a-6470-435d-bc90-4fc19010ca17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:50.101549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8553a0a-6470-435d-bc90-4fc19010ca17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:50.101935 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8553a0a-6470-435d-bc90-4fc19010ca17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:50.102210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8553a0a-6470-435d-bc90-4fc19010ca17 admin admin] PATH_INFO: `/v3/users/2b69dd6e1f7c4d93af3a27f310a7704c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:50.103410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8553a0a-6470-435d-bc90-4fc19010ca17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b69dd6e1f7c4d93af3a27f310a7704c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:50.106868 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8553a0a-6470-435d-bc90-4fc19010ca17 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:50.125102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:50.141950 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3293920-3968-42a8-b162-20ac8db25641 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:50.149690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e80e5-6c9b-440f-a611-fb96d27a1266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:50.150412 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e80e5-6c9b-440f-a611-fb96d27a1266 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:50.151280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e80e5-6c9b-440f-a611-fb96d27a1266 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:50.151551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e80e5-6c9b-440f-a611-fb96d27a1266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:50.164051 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e80e5-6c9b-440f-a611-fb96d27a1266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:50.165511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e80e5-6c9b-440f-a611-fb96d27a1266 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:50.176082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a8553a0a-6470-435d-bc90-4fc19010ca17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b69dd6e1f7c4d93af3a27f310a7704c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:50.177127 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1095/2186] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:50.178726 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e80e5-6c9b-440f-a611-fb96d27a1266 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:50.188611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a8553a0a-6470-435d-bc90-4fc19010ca17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b69dd6e1f7c4d93af3a27f310a7704c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:50.188804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e80e5-6c9b-440f-a611-fb96d27a1266 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:50.193312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c12cdb-3c12-45c7-b35a-1474c10f3b9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:50.193917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c12cdb-3c12-45c7-b35a-1474c10f3b9b service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:50.194197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c12cdb-3c12-45c7-b35a-1474c10f3b9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:50.194512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c12cdb-3c12-45c7-b35a-1474c10f3b9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:50.200185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a8553a0a-6470-435d-bc90-4fc19010ca17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b69dd6e1f7c4d93af3a27f310a7704c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:50.201116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a8553a0a-6470-435d-bc90-4fc19010ca17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b69dd6e1f7c4d93af3a27f310a7704c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:50.202414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a8553a0a-6470-435d-bc90-4fc19010ca17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b69dd6e1f7c4d93af3a27f310a7704c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:50.205630 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c12cdb-3c12-45c7-b35a-1474c10f3b9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:50.207102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c12cdb-3c12-45c7-b35a-1474c10f3b9b service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:50.218143 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:50.220124 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1096/2187] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:50.230909 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1092/2188] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:50 2022] DELETE /identity/v3/users/2b69dd6e1f7c4d93af3a27f310a7704c => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:50.235983 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d7449-6086-447e-8c53-f9e4d9b5535e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:50.236598 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d7449-6086-447e-8c53-f9e4d9b5535e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:50.236775 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d7449-6086-447e-8c53-f9e4d9b5535e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:50.258612 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-2b5d7449-6086-447e-8c53-f9e4d9b5535e None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:50.260122 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2b5d7449-6086-447e-8c53-f9e4d9b5535e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:50.349115 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1097/2189] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:50.368038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8553a0a-6470-435d-bc90-4fc19010ca17 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:50.369497 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a8553a0a-6470-435d-bc90-4fc19010ca17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:50.441428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8553a0a-6470-435d-bc90-4fc19010ca17 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:50.447759 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dee165-f12f-498a-be41-d584aa2ddbe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:50.448468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dee165-f12f-498a-be41-d584aa2ddbe9 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:50.448824 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dee165-f12f-498a-be41-d584aa2ddbe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:50.449171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dee165-f12f-498a-be41-d584aa2ddbe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:50.468937 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dee165-f12f-498a-be41-d584aa2ddbe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:50.470673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dee165-f12f-498a-be41-d584aa2ddbe9 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:50.486178 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1093/2190] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:27:51.013705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d7449-6086-447e-8c53-f9e4d9b5535e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:51.015931 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2b5d7449-6086-447e-8c53-f9e4d9b5535e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:51.034060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d7449-6086-447e-8c53-f9e4d9b5535e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:51.055078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e162fe2-4d9f-479a-90c3-504c04c851a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:51.055749 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e162fe2-4d9f-479a-90c3-504c04c851a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:51.056045 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e162fe2-4d9f-479a-90c3-504c04c851a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:51.056436 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e162fe2-4d9f-479a-90c3-504c04c851a7 admin admin] PATH_INFO: `/v3/projects/00b0a7a94f64457ab14a90ea1ab5f593` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:51.071632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e162fe2-4d9f-479a-90c3-504c04c851a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00b0a7a94f64457ab14a90ea1ab5f593)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:51.071632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e162fe2-4d9f-479a-90c3-504c04c851a7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:51.073562 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dee165-f12f-498a-be41-d584aa2ddbe9 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:51.075224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e0dee165-f12f-498a-be41-d584aa2ddbe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:51.089761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dee165-f12f-498a-be41-d584aa2ddbe9 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:51.096747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f347c7-f953-4c48-aae2-2289233d32fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:51.096747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f347c7-f953-4c48-aae2-2289233d32fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:51.096747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f347c7-f953-4c48-aae2-2289233d32fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:51.096747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f347c7-f953-4c48-aae2-2289233d32fb admin admin] PATH_INFO: `/v3/projects/066baf4af3be4dbc89e510a401ec573b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:51.097362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f347c7-f953-4c48-aae2-2289233d32fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=066baf4af3be4dbc89e510a401ec573b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:51.099504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f347c7-f953-4c48-aae2-2289233d32fb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:51.138178 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9e162fe2-4d9f-479a-90c3-504c04c851a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00b0a7a94f64457ab14a90ea1ab5f593', 'request_id': 'req-9e162fe2-4d9f-479a-90c3-504c04c851a7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.144971 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9e162fe2-4d9f-479a-90c3-504c04c851a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00b0a7a94f64457ab14a90ea1ab5f593', 'request_id': 'req-9e162fe2-4d9f-479a-90c3-504c04c851a7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.155904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9e162fe2-4d9f-479a-90c3-504c04c851a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00b0a7a94f64457ab14a90ea1ab5f593', 'request_id': 'req-9e162fe2-4d9f-479a-90c3-504c04c851a7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.164400 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-82f347c7-f953-4c48-aae2-2289233d32fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '066baf4af3be4dbc89e510a401ec573b', 'request_id': 'req-82f347c7-f953-4c48-aae2-2289233d32fb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.169263 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9e162fe2-4d9f-479a-90c3-504c04c851a7 admin admin] The token cache is being invalidate because project 00b0a7a94f64457ab14a90ea1ab5f593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:51.169592 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9e162fe2-4d9f-479a-90c3-504c04c851a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.170343 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9e162fe2-4d9f-479a-90c3-504c04c851a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.171378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-82f347c7-f953-4c48-aae2-2289233d32fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '066baf4af3be4dbc89e510a401ec573b', 'request_id': 'req-82f347c7-f953-4c48-aae2-2289233d32fb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.172148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1098/2191] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:51 2022] DELETE /identity/v3/projects/00b0a7a94f64457ab14a90ea1ab5f593 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:51.172719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e162fe2-4d9f-479a-90c3-504c04c851a7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:51.174067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9e162fe2-4d9f-479a-90c3-504c04c851a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:51.183649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-82f347c7-f953-4c48-aae2-2289233d32fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '066baf4af3be4dbc89e510a401ec573b', 'request_id': 'req-82f347c7-f953-4c48-aae2-2289233d32fb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.196363 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-82f347c7-f953-4c48-aae2-2289233d32fb admin admin] The token cache is being invalidate because project 066baf4af3be4dbc89e510a401ec573b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:51.196666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-82f347c7-f953-4c48-aae2-2289233d32fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.198175 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-82f347c7-f953-4c48-aae2-2289233d32fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.199598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1094/2192] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:51 2022] DELETE /identity/v3/projects/066baf4af3be4dbc89e510a401ec573b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:51.200300 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f347c7-f953-4c48-aae2-2289233d32fb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:51.201597 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-82f347c7-f953-4c48-aae2-2289233d32fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:51.234865 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:51.250124 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e162fe2-4d9f-479a-90c3-504c04c851a7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:51.267899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f347c7-f953-4c48-aae2-2289233d32fb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:51.272480 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99d727-ecde-4210-935d-79ce57179353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:51.273127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99d727-ecde-4210-935d-79ce57179353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:51.273402 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99d727-ecde-4210-935d-79ce57179353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:51.273669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99d727-ecde-4210-935d-79ce57179353 admin admin] PATH_INFO: `/v3/users/65623a8aefc747d98cf63d109c990fe0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:51.274851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99d727-ecde-4210-935d-79ce57179353 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65623a8aefc747d98cf63d109c990fe0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:51.277332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99d727-ecde-4210-935d-79ce57179353 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:51.299361 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:51.309631 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34405b29-f133-4f10-b233-7555e429fe7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:51.310443 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34405b29-f133-4f10-b233-7555e429fe7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:51.310743 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34405b29-f133-4f10-b233-7555e429fe7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:51.311033 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34405b29-f133-4f10-b233-7555e429fe7b admin admin] PATH_INFO: `/v3/projects/c4a56db115f64261ae4b472567a2265e` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:51.312277 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34405b29-f133-4f10-b233-7555e429fe7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4a56db115f64261ae4b472567a2265e)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:51.314183 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34405b29-f133-4f10-b233-7555e429fe7b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:51.326381 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eb99d727-ecde-4210-935d-79ce57179353 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65623a8aefc747d98cf63d109c990fe0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.334000 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eb99d727-ecde-4210-935d-79ce57179353 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65623a8aefc747d98cf63d109c990fe0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.343180 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eb99d727-ecde-4210-935d-79ce57179353 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65623a8aefc747d98cf63d109c990fe0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.343941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eb99d727-ecde-4210-935d-79ce57179353 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65623a8aefc747d98cf63d109c990fe0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.344536 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eb99d727-ecde-4210-935d-79ce57179353 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65623a8aefc747d98cf63d109c990fe0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.363891 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-34405b29-f133-4f10-b233-7555e429fe7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4a56db115f64261ae4b472567a2265e', 'request_id': 'req-34405b29-f133-4f10-b233-7555e429fe7b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.364857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1095/2193] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:51 2022] DELETE /identity/v3/users/65623a8aefc747d98cf63d109c990fe0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:51.365675 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99d727-ecde-4210-935d-79ce57179353 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:51.366991 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-eb99d727-ecde-4210-935d-79ce57179353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:51.373613 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-34405b29-f133-4f10-b233-7555e429fe7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4a56db115f64261ae4b472567a2265e', 'request_id': 'req-34405b29-f133-4f10-b233-7555e429fe7b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.384036 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-34405b29-f133-4f10-b233-7555e429fe7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4a56db115f64261ae4b472567a2265e', 'request_id': 'req-34405b29-f133-4f10-b233-7555e429fe7b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.399639 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-34405b29-f133-4f10-b233-7555e429fe7b admin admin] The token cache is being invalidate because project c4a56db115f64261ae4b472567a2265e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:51.399894 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-34405b29-f133-4f10-b233-7555e429fe7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.400640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-34405b29-f133-4f10-b233-7555e429fe7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.403201 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1099/2194] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:51 2022] DELETE /identity/v3/projects/c4a56db115f64261ae4b472567a2265e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:51.408299 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c1c49-82c9-4994-85e6-da1cc923ae9c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:51.408589 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c1c49-82c9-4994-85e6-da1cc923ae9c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:51.408858 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c1c49-82c9-4994-85e6-da1cc923ae9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:51.439245 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-652c1c49-82c9-4994-85e6-da1cc923ae9c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:51.445292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-652c1c49-82c9-4994-85e6-da1cc923ae9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:51.454438 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99d727-ecde-4210-935d-79ce57179353 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:51.462795 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1100/2195] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:51.463003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ac99b-7057-4d59-a541-d43205ed7c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:51.463550 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ac99b-7057-4d59-a541-d43205ed7c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:51.463829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ac99b-7057-4d59-a541-d43205ed7c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:51.464101 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ac99b-7057-4d59-a541-d43205ed7c7a admin admin] PATH_INFO: `/v3/users/77eb133292ad405bb9e1eb37c2eb2a48` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:51.465812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ac99b-7057-4d59-a541-d43205ed7c7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=77eb133292ad405bb9e1eb37c2eb2a48)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:51.468243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ac99b-7057-4d59-a541-d43205ed7c7a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:51.470459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a182fa7-67cc-44c0-96c5-5c4efb0e4f41 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:51.471057 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a182fa7-67cc-44c0-96c5-5c4efb0e4f41 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:51.471475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a182fa7-67cc-44c0-96c5-5c4efb0e4f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:51.493963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6a182fa7-67cc-44c0-96c5-5c4efb0e4f41 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:51.495946 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6a182fa7-67cc-44c0-96c5-5c4efb0e4f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:51.505664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1101/2196] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:27:51.509746 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a182fa7-67cc-44c0-96c5-5c4efb0e4f41 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:51.520396 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a182fa7-67cc-44c0-96c5-5c4efb0e4f41 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:51.525607 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af55f4-3d00-46a0-8195-200ca930f38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:51.526274 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af55f4-3d00-46a0-8195-200ca930f38e admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:51.526573 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af55f4-3d00-46a0-8195-200ca930f38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:51.526849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af55f4-3d00-46a0-8195-200ca930f38e admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:51.527510 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af55f4-3d00-46a0-8195-200ca930f38e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:27:51.527821 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af55f4-3d00-46a0-8195-200ca930f38e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:51.529106 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ff4ac99b-7057-4d59-a541-d43205ed7c7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77eb133292ad405bb9e1eb37c2eb2a48'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.529600 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af55f4-3d00-46a0-8195-200ca930f38e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:51.540782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ff4ac99b-7057-4d59-a541-d43205ed7c7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77eb133292ad405bb9e1eb37c2eb2a48'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.547225 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1102/2197] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:27:51.548364 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af55f4-3d00-46a0-8195-200ca930f38e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:51.551278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ff4ac99b-7057-4d59-a541-d43205ed7c7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77eb133292ad405bb9e1eb37c2eb2a48'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.552620 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ff4ac99b-7057-4d59-a541-d43205ed7c7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77eb133292ad405bb9e1eb37c2eb2a48'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.554286 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ff4ac99b-7057-4d59-a541-d43205ed7c7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77eb133292ad405bb9e1eb37c2eb2a48'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.559744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af55f4-3d00-46a0-8195-200ca930f38e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:51.565344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f905a9-717c-4f5a-97b0-aa64768d872f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:51.566161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f905a9-717c-4f5a-97b0-aa64768d872f admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:51.566666 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f905a9-717c-4f5a-97b0-aa64768d872f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:51.567116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f905a9-717c-4f5a-97b0-aa64768d872f admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:51.568378 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f905a9-717c-4f5a-97b0-aa64768d872f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:51.570930 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f905a9-717c-4f5a-97b0-aa64768d872f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:51.599900 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1096/2198] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:51 2022] DELETE /identity/v3/users/77eb133292ad405bb9e1eb37c2eb2a48 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:51.610395 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1103/2199] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:51 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:51.614840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ac99b-7057-4d59-a541-d43205ed7c7a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:51.616515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ff4ac99b-7057-4d59-a541-d43205ed7c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:51.644425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:51.710723 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ac99b-7057-4d59-a541-d43205ed7c7a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:51.801304 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f003f-7278-40c4-87c7-e0f7132446a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:51.802738 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f003f-7278-40c4-87c7-e0f7132446a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:51.803139 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f003f-7278-40c4-87c7-e0f7132446a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:51.803602 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f003f-7278-40c4-87c7-e0f7132446a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:51.804441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f003f-7278-40c4-87c7-e0f7132446a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:51.806658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f003f-7278-40c4-87c7-e0f7132446a0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:51.827030 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f905a9-717c-4f5a-97b0-aa64768d872f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:51.830072 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b4f905a9-717c-4f5a-97b0-aa64768d872f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:51.842826 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1097/2200] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:51 2022] POST /identity/v3/users => generated 292 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:27:51.847344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f003f-7278-40c4-87c7-e0f7132446a0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:51.852860 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f905a9-717c-4f5a-97b0-aa64768d872f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:51.864702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f003f-7278-40c4-87c7-e0f7132446a0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:51.868643 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a6eff-c567-4eaa-9785-4c8b62ca6efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:51.869298 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a6eff-c567-4eaa-9785-4c8b62ca6efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:51.870197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a6eff-c567-4eaa-9785-4c8b62ca6efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:51.870504 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a6eff-c567-4eaa-9785-4c8b62ca6efa admin admin] PATH_INFO: `/v3/projects/34464e76b222459396c9dce08646e224` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:51.876927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a6eff-c567-4eaa-9785-4c8b62ca6efa admin admin] RBAC: Authorizing `identity:delete_project(project_id=34464e76b222459396c9dce08646e224)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:51.878393 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764a952-c32c-41a1-b91a-29e89e29b883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:51.879428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764a952-c32c-41a1-b91a-29e89e29b883 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:51.879554 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a6eff-c567-4eaa-9785-4c8b62ca6efa admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:51.879675 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764a952-c32c-41a1-b91a-29e89e29b883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:51.880065 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764a952-c32c-41a1-b91a-29e89e29b883 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:51.881049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764a952-c32c-41a1-b91a-29e89e29b883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:51.894270 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764a952-c32c-41a1-b91a-29e89e29b883 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:51.910143 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1098/2201] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:51.912158 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764a952-c32c-41a1-b91a-29e89e29b883 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:51.926426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764a952-c32c-41a1-b91a-29e89e29b883 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:51.933985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499293b6-eabf-4762-af32-045ac47e857f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:51.934704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499293b6-eabf-4762-af32-045ac47e857f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:51.935141 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499293b6-eabf-4762-af32-045ac47e857f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:51.935407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499293b6-eabf-4762-af32-045ac47e857f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:51.936130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499293b6-eabf-4762-af32-045ac47e857f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:51.937910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499293b6-eabf-4762-af32-045ac47e857f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:51.947653 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:51.951608 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1099/2202] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:51.955682 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499293b6-eabf-4762-af32-045ac47e857f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:51.962741 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3a3a6eff-c567-4eaa-9785-4c8b62ca6efa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34464e76b222459396c9dce08646e224', 'request_id': 'req-3a3a6eff-c567-4eaa-9785-4c8b62ca6efa'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.968075 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499293b6-eabf-4762-af32-045ac47e857f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:51.970097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3a3a6eff-c567-4eaa-9785-4c8b62ca6efa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34464e76b222459396c9dce08646e224', 'request_id': 'req-3a3a6eff-c567-4eaa-9785-4c8b62ca6efa'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.982365 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3a3a6eff-c567-4eaa-9785-4c8b62ca6efa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34464e76b222459396c9dce08646e224', 'request_id': 'req-3a3a6eff-c567-4eaa-9785-4c8b62ca6efa'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.995829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3a3a6eff-c567-4eaa-9785-4c8b62ca6efa admin admin] The token cache is being invalidate because project 34464e76b222459396c9dce08646e224 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:51.996264 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3a3a6eff-c567-4eaa-9785-4c8b62ca6efa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.996939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3a3a6eff-c567-4eaa-9785-4c8b62ca6efa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.998675 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1104/2203] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:51 2022] DELETE /identity/v3/projects/34464e76b222459396c9dce08646e224 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:27:52.003978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc6208-70d9-45cd-a14a-db50af3f1cb0 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.004319 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc6208-70d9-45cd-a14a-db50af3f1cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.004578 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc6208-70d9-45cd-a14a-db50af3f1cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.025288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-51dc6208-70d9-45cd-a14a-db50af3f1cb0 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:52.027640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-51dc6208-70d9-45cd-a14a-db50af3f1cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:52.046110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb43909-3217-4f03-93b6-4a61462b0915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.047049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb43909-3217-4f03-93b6-4a61462b0915 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.047377 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb43909-3217-4f03-93b6-4a61462b0915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.047656 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb43909-3217-4f03-93b6-4a61462b0915 admin admin] PATH_INFO: `/v3/projects/2d75f4c0fe9148e7bf633a79dbfd6bb9/users/ef2ca1a30a76468cbe02b20efd6ca37e/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.065205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb43909-3217-4f03-93b6-4a61462b0915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d75f4c0fe9148e7bf633a79dbfd6bb9, user_id=ef2ca1a30a76468cbe02b20efd6ca37e, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.067998 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb43909-3217-4f03-93b6-4a61462b0915 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.084159 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1100/2204] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:51 2022] PUT /identity/v3/projects/2d75f4c0fe9148e7bf633a79dbfd6bb9/users/ef2ca1a30a76468cbe02b20efd6ca37e/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:27:52.085277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb43909-3217-4f03-93b6-4a61462b0915 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.088732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5bb43909-3217-4f03-93b6-4a61462b0915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:52.092362 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1105/2205] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:52.093525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecb9c6-0992-41d6-b8f4-fec8d0a30894 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.093809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecb9c6-0992-41d6-b8f4-fec8d0a30894 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.094082 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecb9c6-0992-41d6-b8f4-fec8d0a30894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.115693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c5ecb9c6-0992-41d6-b8f4-fec8d0a30894 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:52.116934 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c5ecb9c6-0992-41d6-b8f4-fec8d0a30894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:52.118030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:52.169980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb43909-3217-4f03-93b6-4a61462b0915 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.176748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348f2e2-86ac-4aff-a697-1f5ff1ee33c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.177876 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348f2e2-86ac-4aff-a697-1f5ff1ee33c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.178320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348f2e2-86ac-4aff-a697-1f5ff1ee33c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.178794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348f2e2-86ac-4aff-a697-1f5ff1ee33c6 admin admin] PATH_INFO: `/v3/projects/9dcdcb70aeea49f190dcfb60c5687942` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.180292 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348f2e2-86ac-4aff-a697-1f5ff1ee33c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dcdcb70aeea49f190dcfb60c5687942)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.182450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348f2e2-86ac-4aff-a697-1f5ff1ee33c6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.191830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1106/2206] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:52.193184 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9ec19-1fde-4400-93fb-a0913f31cda1 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.193474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9ec19-1fde-4400-93fb-a0913f31cda1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.193754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9ec19-1fde-4400-93fb-a0913f31cda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.216746 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-53f9ec19-1fde-4400-93fb-a0913f31cda1 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:52.217906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-53f9ec19-1fde-4400-93fb-a0913f31cda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:52.236984 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e348f2e2-86ac-4aff-a697-1f5ff1ee33c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dcdcb70aeea49f190dcfb60c5687942', 'request_id': 'req-e348f2e2-86ac-4aff-a697-1f5ff1ee33c6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:52.243002 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e348f2e2-86ac-4aff-a697-1f5ff1ee33c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dcdcb70aeea49f190dcfb60c5687942', 'request_id': 'req-e348f2e2-86ac-4aff-a697-1f5ff1ee33c6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:52.259516 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e348f2e2-86ac-4aff-a697-1f5ff1ee33c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dcdcb70aeea49f190dcfb60c5687942', 'request_id': 'req-e348f2e2-86ac-4aff-a697-1f5ff1ee33c6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:52.272398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e348f2e2-86ac-4aff-a697-1f5ff1ee33c6 admin admin] The token cache is being invalidate because project 9dcdcb70aeea49f190dcfb60c5687942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:52.272714 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e348f2e2-86ac-4aff-a697-1f5ff1ee33c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:52.273366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e348f2e2-86ac-4aff-a697-1f5ff1ee33c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:52.274874 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1101/2207] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:52 2022] DELETE /identity/v3/projects/9dcdcb70aeea49f190dcfb60c5687942 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:52.276244 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348f2e2-86ac-4aff-a697-1f5ff1ee33c6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.279562 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e348f2e2-86ac-4aff-a697-1f5ff1ee33c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:52.283933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1107/2208] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:52.285180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc5e2a-b6b3-48ee-9ed5-dd7859ce8ef7 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.285466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc5e2a-b6b3-48ee-9ed5-dd7859ce8ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.285741 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc5e2a-b6b3-48ee-9ed5-dd7859ce8ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.305845 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:52.315675 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-defc5e2a-b6b3-48ee-9ed5-dd7859ce8ef7 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:52.317812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-defc5e2a-b6b3-48ee-9ed5-dd7859ce8ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:52.325986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1108/2209] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:52.332654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc5e2a-b6b3-48ee-9ed5-dd7859ce8ef7 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.342563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc5e2a-b6b3-48ee-9ed5-dd7859ce8ef7 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.347433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348f2e2-86ac-4aff-a697-1f5ff1ee33c6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.347883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f89ce-4b50-48a9-8502-bafc8ce19e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.348474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f89ce-4b50-48a9-8502-bafc8ce19e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.348752 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f89ce-4b50-48a9-8502-bafc8ce19e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.349021 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f89ce-4b50-48a9-8502-bafc8ce19e8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.349694 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f89ce-4b50-48a9-8502-bafc8ce19e8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:27:52.349997 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f89ce-4b50-48a9-8502-bafc8ce19e8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.351181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc4fed-c037-4df7-9b9b-6403a1fad0fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.351787 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc4fed-c037-4df7-9b9b-6403a1fad0fa service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.352066 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc4fed-c037-4df7-9b9b-6403a1fad0fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.352176 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f89ce-4b50-48a9-8502-bafc8ce19e8b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.352340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc4fed-c037-4df7-9b9b-6403a1fad0fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.354770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5bbc4fed-c037-4df7-9b9b-6403a1fad0fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:52.369409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1109/2210] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:27:52.369674 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc4fed-c037-4df7-9b9b-6403a1fad0fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.369777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04c16b-bae0-4642-83a2-e46cc0730f74 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.369777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04c16b-bae0-4642-83a2-e46cc0730f74 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.369777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04c16b-bae0-4642-83a2-e46cc0730f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.370025 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc4fed-c037-4df7-9b9b-6403a1fad0fa service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.378928 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1102/2211] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:52.379605 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc4fed-c037-4df7-9b9b-6403a1fad0fa service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.389894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc4fed-c037-4df7-9b9b-6403a1fad0fa service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.390027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-2f04c16b-bae0-4642-83a2-e46cc0730f74 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:52.391206 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2f04c16b-bae0-4642-83a2-e46cc0730f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:52.394914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fad40-f311-4297-a54b-1f8e53d1609a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.395637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fad40-f311-4297-a54b-1f8e53d1609a admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.395923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fad40-f311-4297-a54b-1f8e53d1609a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.396198 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fad40-f311-4297-a54b-1f8e53d1609a admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.396826 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0fad40-f311-4297-a54b-1f8e53d1609a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.398774 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0fad40-f311-4297-a54b-1f8e53d1609a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.401395 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1110/2212] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:52.406534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04c16b-bae0-4642-83a2-e46cc0730f74 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.415874 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04c16b-bae0-4642-83a2-e46cc0730f74 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.424704 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0c426-0247-40e0-bccf-63feb27a4a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.425778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0c426-0247-40e0-bccf-63feb27a4a1e admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.425778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0c426-0247-40e0-bccf-63feb27a4a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.426324 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0c426-0247-40e0-bccf-63feb27a4a1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.427074 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f0c426-0247-40e0-bccf-63feb27a4a1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:27:52.427472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f0c426-0247-40e0-bccf-63feb27a4a1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.429449 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f0c426-0247-40e0-bccf-63feb27a4a1e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.450794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1111/2213] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:27:52.455190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0c426-0247-40e0-bccf-63feb27a4a1e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.460847 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1103/2214] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:52 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:52.461827 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fad40-f311-4297-a54b-1f8e53d1609a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.470726 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0c426-0247-40e0-bccf-63feb27a4a1e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.508299 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:52.556177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870edec-5a8f-4769-8be1-cfd34dc73eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.556857 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870edec-5a8f-4769-8be1-cfd34dc73eea admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.557204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870edec-5a8f-4769-8be1-cfd34dc73eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.557457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870edec-5a8f-4769-8be1-cfd34dc73eea admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.558117 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870edec-5a8f-4769-8be1-cfd34dc73eea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.560170 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870edec-5a8f-4769-8be1-cfd34dc73eea admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.562143 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fad40-f311-4297-a54b-1f8e53d1609a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.568386 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650533e8-f3f4-4e70-8170-95aa601477c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.569003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650533e8-f3f4-4e70-8170-95aa601477c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.569279 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650533e8-f3f4-4e70-8170-95aa601477c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.569712 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650533e8-f3f4-4e70-8170-95aa601477c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.570488 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650533e8-f3f4-4e70-8170-95aa601477c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.572416 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650533e8-f3f4-4e70-8170-95aa601477c9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.596836 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1104/2215] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:52 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:52.599192 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1112/2216] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:52 2022] POST /identity/v3/projects => generated 361 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:27:52.601119 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650533e8-f3f4-4e70-8170-95aa601477c9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.604990 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870edec-5a8f-4769-8be1-cfd34dc73eea admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.617864 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650533e8-f3f4-4e70-8170-95aa601477c9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.679856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870edec-5a8f-4769-8be1-cfd34dc73eea admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.684635 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edc049-72d6-479a-8aeb-9873b06acb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.685247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edc049-72d6-479a-8aeb-9873b06acb5a admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.685536 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edc049-72d6-479a-8aeb-9873b06acb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.685791 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edc049-72d6-479a-8aeb-9873b06acb5a admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.686471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97edc049-72d6-479a-8aeb-9873b06acb5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.688613 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97edc049-72d6-479a-8aeb-9873b06acb5a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.689911 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3c6bc-1ca9-4a4a-9dad-7f10b5fd7ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.690843 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3c6bc-1ca9-4a4a-9dad-7f10b5fd7ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.690965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3c6bc-1ca9-4a4a-9dad-7f10b5fd7ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.691312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3c6bc-1ca9-4a4a-9dad-7f10b5fd7ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.691983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3c6bc-1ca9-4a4a-9dad-7f10b5fd7ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.693770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3c6bc-1ca9-4a4a-9dad-7f10b5fd7ed0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.706134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1105/2217] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:52.713034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3c6bc-1ca9-4a4a-9dad-7f10b5fd7ed0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.721466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1113/2218] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:52 2022] POST /identity/v3/users => generated 281 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:52.722456 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edc049-72d6-479a-8aeb-9873b06acb5a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.724176 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3c6bc-1ca9-4a4a-9dad-7f10b5fd7ed0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.729376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc156e-5bf6-411f-b2ab-46e8f7a69ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.729792 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc156e-5bf6-411f-b2ab-46e8f7a69ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.730073 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc156e-5bf6-411f-b2ab-46e8f7a69ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.730398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc156e-5bf6-411f-b2ab-46e8f7a69ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.731013 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc156e-5bf6-411f-b2ab-46e8f7a69ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.732725 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc156e-5bf6-411f-b2ab-46e8f7a69ec4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.734414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edc049-72d6-479a-8aeb-9873b06acb5a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.740478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda42bd-09f0-4954-a51d-47e857c9de69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.741310 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda42bd-09f0-4954-a51d-47e857c9de69 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.741615 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda42bd-09f0-4954-a51d-47e857c9de69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.742125 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda42bd-09f0-4954-a51d-47e857c9de69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.743631 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda42bd-09f0-4954-a51d-47e857c9de69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.746085 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda42bd-09f0-4954-a51d-47e857c9de69 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.746522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1106/2219] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:27:52.750156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc156e-5bf6-411f-b2ab-46e8f7a69ec4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.764280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1114/2220] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:52.766419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc156e-5bf6-411f-b2ab-46e8f7a69ec4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.768114 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda42bd-09f0-4954-a51d-47e857c9de69 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.771810 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962543a8-881a-425b-85dd-71ffaa00b520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.772186 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962543a8-881a-425b-85dd-71ffaa00b520 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.772583 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962543a8-881a-425b-85dd-71ffaa00b520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.772920 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962543a8-881a-425b-85dd-71ffaa00b520 admin admin] PATH_INFO: `/v3/projects/0128bb18d7f945d39adb6ae5031a0b74/users/ccc07ee0ae9f4442852185d37ff5015d/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.780132 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda42bd-09f0-4954-a51d-47e857c9de69 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.787357 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a290d-e7ea-41db-b2db-b4c07d840145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.788554 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a290d-e7ea-41db-b2db-b4c07d840145 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.788554 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a290d-e7ea-41db-b2db-b4c07d840145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.789428 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a290d-e7ea-41db-b2db-b4c07d840145 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.790848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a290d-e7ea-41db-b2db-b4c07d840145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.792501 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a290d-e7ea-41db-b2db-b4c07d840145 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.795081 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962543a8-881a-425b-85dd-71ffaa00b520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0128bb18d7f945d39adb6ae5031a0b74, user_id=ccc07ee0ae9f4442852185d37ff5015d, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.798199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962543a8-881a-425b-85dd-71ffaa00b520 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.806042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1115/2221] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:52.810815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a290d-e7ea-41db-b2db-b4c07d840145 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.817605 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1107/2222] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:52 2022] PUT /identity/v3/projects/0128bb18d7f945d39adb6ae5031a0b74/users/ccc07ee0ae9f4442852185d37ff5015d/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:27:52.822303 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a290d-e7ea-41db-b2db-b4c07d840145 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.829558 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e6401-50e7-46f9-847e-b05751732241 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.830823 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e6401-50e7-46f9-847e-b05751732241 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.831265 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e6401-50e7-46f9-847e-b05751732241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.852345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:52.870456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-113e6401-50e7-46f9-847e-b05751732241 None None] MFA Rules not processed for user `ccc07ee0ae9f4442852185d37ff5015d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:52.871769 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-113e6401-50e7-46f9-847e-b05751732241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:52.911808 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1033b-7ebb-44bb-904b-3fa6be5309ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.912478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1033b-7ebb-44bb-904b-3fa6be5309ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.912762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1033b-7ebb-44bb-904b-3fa6be5309ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.913040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1033b-7ebb-44bb-904b-3fa6be5309ef admin admin] PATH_INFO: `/v3/projects/5dc7e7c7f5344d489eade7be31af943c/users/44f6ef9b641b4a46b7ebc06eae344d72/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.940448 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d1033b-7ebb-44bb-904b-3fa6be5309ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc7e7c7f5344d489eade7be31af943c, user_id=44f6ef9b641b4a46b7ebc06eae344d72, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.944101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d1033b-7ebb-44bb-904b-3fa6be5309ef admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.956567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1108/2223] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:52.969742 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e6401-50e7-46f9-847e-b05751732241 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.978683 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1116/2224] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:52 2022] PUT /identity/v3/projects/5dc7e7c7f5344d489eade7be31af943c/users/44f6ef9b641b4a46b7ebc06eae344d72/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:52.979972 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-113e6401-50e7-46f9-847e-b05751732241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:52.987091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d085c8-86d6-49f2-a0d8-fd91613aeeb4 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.987591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d085c8-86d6-49f2-a0d8-fd91613aeeb4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.988175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d085c8-86d6-49f2-a0d8-fd91613aeeb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.993503 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:53.023730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-27d085c8-86d6-49f2-a0d8-fd91613aeeb4 None None] MFA Rules not processed for user `44f6ef9b641b4a46b7ebc06eae344d72`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:53.025243 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-27d085c8-86d6-49f2-a0d8-fd91613aeeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:53.060756 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e6401-50e7-46f9-847e-b05751732241 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:53.111058 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1117/2225] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:53.142722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d085c8-86d6-49f2-a0d8-fd91613aeeb4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:53.144157 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-27d085c8-86d6-49f2-a0d8-fd91613aeeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:53.145735 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2e4df-489e-4fef-ac07-050235145a47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:53.146844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2e4df-489e-4fef-ac07-050235145a47 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:53.147126 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2e4df-489e-4fef-ac07-050235145a47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:53.147399 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2e4df-489e-4fef-ac07-050235145a47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:53.158037 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d085c8-86d6-49f2-a0d8-fd91613aeeb4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:53.163108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81cda4-e852-4712-82d5-82315cc418fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:53.164001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81cda4-e852-4712-82d5-82315cc418fd service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:53.164451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81cda4-e852-4712-82d5-82315cc418fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:53.164865 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81cda4-e852-4712-82d5-82315cc418fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:53.190290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81cda4-e852-4712-82d5-82315cc418fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:53.193019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81cda4-e852-4712-82d5-82315cc418fd service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:53.218683 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1118/2226] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:53.244477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81cda4-e852-4712-82d5-82315cc418fd service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:53.272320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2e4df-489e-4fef-ac07-050235145a47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:53.274881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2e4df-489e-4fef-ac07-050235145a47 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:53.289646 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1109/2227] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:53.292106 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49d257-bee1-4c6e-8e5c-8a48890e9394 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:53.292390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49d257-bee1-4c6e-8e5c-8a48890e9394 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:53.292661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49d257-bee1-4c6e-8e5c-8a48890e9394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:53.325872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81cda4-e852-4712-82d5-82315cc418fd service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:53.331144 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bd49d257-bee1-4c6e-8e5c-8a48890e9394 None None] MFA Rules not processed for user `ef2ca1a30a76468cbe02b20efd6ca37e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:53.331899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fdb49-8e18-4cfd-a1fa-66a533af5330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:53.332418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bd49d257-bee1-4c6e-8e5c-8a48890e9394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:53.332732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fdb49-8e18-4cfd-a1fa-66a533af5330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:53.333113 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fdb49-8e18-4cfd-a1fa-66a533af5330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:53.333732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fdb49-8e18-4cfd-a1fa-66a533af5330 admin admin] PATH_INFO: `/v3/users/44f6ef9b641b4a46b7ebc06eae344d72` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:53.335229 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fdb49-8e18-4cfd-a1fa-66a533af5330 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44f6ef9b641b4a46b7ebc06eae344d72)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:53.337752 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fdb49-8e18-4cfd-a1fa-66a533af5330 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:53.367670 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:53.400542 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-164fdb49-8e18-4cfd-a1fa-66a533af5330 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44f6ef9b641b4a46b7ebc06eae344d72'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:53.408285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1110/2228] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:53.409080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-164fdb49-8e18-4cfd-a1fa-66a533af5330 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44f6ef9b641b4a46b7ebc06eae344d72'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:53.412200 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49d257-bee1-4c6e-8e5c-8a48890e9394 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:53.424281 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-164fdb49-8e18-4cfd-a1fa-66a533af5330 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44f6ef9b641b4a46b7ebc06eae344d72'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:53.426090 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-164fdb49-8e18-4cfd-a1fa-66a533af5330 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44f6ef9b641b4a46b7ebc06eae344d72'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:53.427084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-164fdb49-8e18-4cfd-a1fa-66a533af5330 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44f6ef9b641b4a46b7ebc06eae344d72'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:53.430985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49d257-bee1-4c6e-8e5c-8a48890e9394 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:53.439969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccdce7-24d7-4c05-9833-ff570fc07a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:53.442761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccdce7-24d7-4c05-9833-ff570fc07a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:53.442761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccdce7-24d7-4c05-9833-ff570fc07a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:53.442761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccdce7-24d7-4c05-9833-ff570fc07a2e admin admin] PATH_INFO: `/v3/users/ccc07ee0ae9f4442852185d37ff5015d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:53.447833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ccdce7-24d7-4c05-9833-ff570fc07a2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccc07ee0ae9f4442852185d37ff5015d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:53.449813 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ccdce7-24d7-4c05-9833-ff570fc07a2e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:53.453624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:53.466675 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1119/2229] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:53 2022] DELETE /identity/v3/users/44f6ef9b641b4a46b7ebc06eae344d72 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:53.467950 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fdb49-8e18-4cfd-a1fa-66a533af5330 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:53.469614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-164fdb49-8e18-4cfd-a1fa-66a533af5330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:53.511696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a7ccdce7-24d7-4c05-9833-ff570fc07a2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccc07ee0ae9f4442852185d37ff5015d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:53.519744 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a7ccdce7-24d7-4c05-9833-ff570fc07a2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccc07ee0ae9f4442852185d37ff5015d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:53.527469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a7ccdce7-24d7-4c05-9833-ff570fc07a2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccc07ee0ae9f4442852185d37ff5015d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:53.528120 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a7ccdce7-24d7-4c05-9833-ff570fc07a2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccc07ee0ae9f4442852185d37ff5015d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:53.528797 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a7ccdce7-24d7-4c05-9833-ff570fc07a2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccc07ee0ae9f4442852185d37ff5015d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:53.533639 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fdb49-8e18-4cfd-a1fa-66a533af5330 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:53.537518 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a85ac3-0b06-4003-ba92-6360b4076701 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:53.538115 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a85ac3-0b06-4003-ba92-6360b4076701 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:53.538425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a85ac3-0b06-4003-ba92-6360b4076701 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:53.538826 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a85ac3-0b06-4003-ba92-6360b4076701 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:53.540548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-11a85ac3-0b06-4003-ba92-6360b4076701 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:53.550127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1111/2230] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:53 2022] DELETE /identity/v3/users/ccc07ee0ae9f4442852185d37ff5015d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:53.550127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea59b26-e0d5-4c4b-9baf-8823c48bfbc4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:53.550127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea59b26-e0d5-4c4b-9baf-8823c48bfbc4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:53.550127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea59b26-e0d5-4c4b-9baf-8823c48bfbc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:53.567516 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fea59b26-e0d5-4c4b-9baf-8823c48bfbc4 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:53.568713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fea59b26-e0d5-4c4b-9baf-8823c48bfbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:53.600496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a85ac3-0b06-4003-ba92-6360b4076701 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:53.601975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a85ac3-0b06-4003-ba92-6360b4076701 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:53.611722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1120/2231] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:53.612973 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9535de1-4409-4a02-b289-c580918c9afe None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:53.613308 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9535de1-4409-4a02-b289-c580918c9afe None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:53.613639 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9535de1-4409-4a02-b289-c580918c9afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:53.634907 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a9535de1-4409-4a02-b289-c580918c9afe None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:53.635268 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1112/2232] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:53.637086 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a9535de1-4409-4a02-b289-c580918c9afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:53.645854 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea59b26-e0d5-4c4b-9baf-8823c48bfbc4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:53.648283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fea59b26-e0d5-4c4b-9baf-8823c48bfbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:53.652258 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1121/2233] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:53.663656 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9535de1-4409-4a02-b289-c580918c9afe None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:53.667081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a9535de1-4409-4a02-b289-c580918c9afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:53.740362 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:53.751795 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9535de1-4409-4a02-b289-c580918c9afe None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:53.758893 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea59b26-e0d5-4c4b-9baf-8823c48bfbc4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:53.760166 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c65236-246e-4812-b36c-fb42f7a09d12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:53.761436 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c65236-246e-4812-b36c-fb42f7a09d12 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:53.761783 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c65236-246e-4812-b36c-fb42f7a09d12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:53.762170 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c65236-246e-4812-b36c-fb42f7a09d12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:53.768745 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738e25f-af8c-4cf2-afda-6474f09995b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:53.769770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738e25f-af8c-4cf2-afda-6474f09995b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:53.770054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738e25f-af8c-4cf2-afda-6474f09995b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:53.770355 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738e25f-af8c-4cf2-afda-6474f09995b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:53.777021 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c65236-246e-4812-b36c-fb42f7a09d12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:53.778541 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c65236-246e-4812-b36c-fb42f7a09d12 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:53.789657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1122/2234] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:53.814696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738e25f-af8c-4cf2-afda-6474f09995b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:53.816415 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738e25f-af8c-4cf2-afda-6474f09995b3 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:53.836033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1113/2235] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:54.314152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c65236-246e-4812-b36c-fb42f7a09d12 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:54.316056 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-92c65236-246e-4812-b36c-fb42f7a09d12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:54.339051 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c65236-246e-4812-b36c-fb42f7a09d12 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:54.344545 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454d51c-04aa-4d29-bf0c-dbb370883fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:54.345252 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454d51c-04aa-4d29-bf0c-dbb370883fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:54.345701 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454d51c-04aa-4d29-bf0c-dbb370883fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:54.345978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454d51c-04aa-4d29-bf0c-dbb370883fd2 admin admin] PATH_INFO: `/v3/projects/5dc7e7c7f5344d489eade7be31af943c` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:54.347876 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738e25f-af8c-4cf2-afda-6474f09995b3 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:54.349133 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454d51c-04aa-4d29-bf0c-dbb370883fd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dc7e7c7f5344d489eade7be31af943c)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:54.349777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e738e25f-af8c-4cf2-afda-6474f09995b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:54.351435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454d51c-04aa-4d29-bf0c-dbb370883fd2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:54.366515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738e25f-af8c-4cf2-afda-6474f09995b3 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:54.373168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451089c-dc33-47de-bae3-deb84b00ef45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:54.374188 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451089c-dc33-47de-bae3-deb84b00ef45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:54.374666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451089c-dc33-47de-bae3-deb84b00ef45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:54.375107 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451089c-dc33-47de-bae3-deb84b00ef45 admin admin] PATH_INFO: `/v3/projects/0128bb18d7f945d39adb6ae5031a0b74` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:54.376657 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451089c-dc33-47de-bae3-deb84b00ef45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0128bb18d7f945d39adb6ae5031a0b74)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:54.379067 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451089c-dc33-47de-bae3-deb84b00ef45 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:54.402127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:54.406496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d454d51c-04aa-4d29-bf0c-dbb370883fd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dc7e7c7f5344d489eade7be31af943c', 'request_id': 'req-d454d51c-04aa-4d29-bf0c-dbb370883fd2'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:54.413226 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d454d51c-04aa-4d29-bf0c-dbb370883fd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dc7e7c7f5344d489eade7be31af943c', 'request_id': 'req-d454d51c-04aa-4d29-bf0c-dbb370883fd2'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:54.422884 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d454d51c-04aa-4d29-bf0c-dbb370883fd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dc7e7c7f5344d489eade7be31af943c', 'request_id': 'req-d454d51c-04aa-4d29-bf0c-dbb370883fd2'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:54.430848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a451089c-dc33-47de-bae3-deb84b00ef45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0128bb18d7f945d39adb6ae5031a0b74', 'request_id': 'req-a451089c-dc33-47de-bae3-deb84b00ef45'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:54.434787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d454d51c-04aa-4d29-bf0c-dbb370883fd2 admin admin] The token cache is being invalidate because project 5dc7e7c7f5344d489eade7be31af943c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:54.435077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d454d51c-04aa-4d29-bf0c-dbb370883fd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:54.435713 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-d454d51c-04aa-4d29-bf0c-dbb370883fd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:54.436944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a451089c-dc33-47de-bae3-deb84b00ef45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0128bb18d7f945d39adb6ae5031a0b74', 'request_id': 'req-a451089c-dc33-47de-bae3-deb84b00ef45'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:54.437209 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1123/2236] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:54 2022] DELETE /identity/v3/projects/5dc7e7c7f5344d489eade7be31af943c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:54.442540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf6b29-d5af-4f42-8083-2d48f7cd9688 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:54.442988 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf6b29-d5af-4f42-8083-2d48f7cd9688 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:54.443276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf6b29-d5af-4f42-8083-2d48f7cd9688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:54.446451 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a451089c-dc33-47de-bae3-deb84b00ef45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0128bb18d7f945d39adb6ae5031a0b74', 'request_id': 'req-a451089c-dc33-47de-bae3-deb84b00ef45'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:54.461365 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a451089c-dc33-47de-bae3-deb84b00ef45 admin admin] The token cache is being invalidate because project 0128bb18d7f945d39adb6ae5031a0b74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:54.461824 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a451089c-dc33-47de-bae3-deb84b00ef45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:54.462500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a451089c-dc33-47de-bae3-deb84b00ef45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:54.464043 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1114/2237] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:54 2022] DELETE /identity/v3/projects/0128bb18d7f945d39adb6ae5031a0b74 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:27:54.471312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-4dcf6b29-d5af-4f42-8083-2d48f7cd9688 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:54.471312 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4dcf6b29-d5af-4f42-8083-2d48f7cd9688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:54.535756 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1124/2238] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:54.544945 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d4c30-0390-44d2-81d3-76bce1da814f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:54.545304 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d4c30-0390-44d2-81d3-76bce1da814f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:54.545504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d4c30-0390-44d2-81d3-76bce1da814f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:54.580585 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-eb0d4c30-0390-44d2-81d3-76bce1da814f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:54.582533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-eb0d4c30-0390-44d2-81d3-76bce1da814f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:54.594288 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1115/2239] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:54.596616 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf6b29-d5af-4f42-8083-2d48f7cd9688 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:54.607095 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf6b29-d5af-4f42-8083-2d48f7cd9688 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:54.612273 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08f11e-feb5-412a-b36b-18a2585b0d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:54.613150 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08f11e-feb5-412a-b36b-18a2585b0d32 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:54.613525 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08f11e-feb5-412a-b36b-18a2585b0d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:54.613891 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08f11e-feb5-412a-b36b-18a2585b0d32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:54.614806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08f11e-feb5-412a-b36b-18a2585b0d32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:27:54.615208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08f11e-feb5-412a-b36b-18a2585b0d32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:54.616945 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08f11e-feb5-412a-b36b-18a2585b0d32 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:54.631854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1125/2240] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:27:54.636530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d4c30-0390-44d2-81d3-76bce1da814f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:54.648005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d4c30-0390-44d2-81d3-76bce1da814f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:54.653285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f2a01-646d-438a-8700-8cd83dfa0c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:54.653910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f2a01-646d-438a-8700-8cd83dfa0c41 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:54.654187 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f2a01-646d-438a-8700-8cd83dfa0c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:54.654492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f2a01-646d-438a-8700-8cd83dfa0c41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:54.655341 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f2a01-646d-438a-8700-8cd83dfa0c41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:54.657105 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f2a01-646d-438a-8700-8cd83dfa0c41 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:54.691812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1116/2241] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:54 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:54.693023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08f11e-feb5-412a-b36b-18a2585b0d32 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:54.706040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08f11e-feb5-412a-b36b-18a2585b0d32 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:54.761139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0499c48-cf40-4b53-b629-b30fc73f7a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:54.761732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0499c48-cf40-4b53-b629-b30fc73f7a34 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:54.762000 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0499c48-cf40-4b53-b629-b30fc73f7a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:54.762291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0499c48-cf40-4b53-b629-b30fc73f7a34 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:54.762935 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0499c48-cf40-4b53-b629-b30fc73f7a34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:54.764829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0499c48-cf40-4b53-b629-b30fc73f7a34 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:54.773715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:54.787527 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1126/2242] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:54 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:54.792132 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f2a01-646d-438a-8700-8cd83dfa0c41 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:54.802442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f2a01-646d-438a-8700-8cd83dfa0c41 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:54.807663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aac579-82ed-40da-bd6d-3d893df12acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:54.808236 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aac579-82ed-40da-bd6d-3d893df12acf admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:54.808510 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aac579-82ed-40da-bd6d-3d893df12acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:54.808770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aac579-82ed-40da-bd6d-3d893df12acf admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:54.809397 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aac579-82ed-40da-bd6d-3d893df12acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:54.811015 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aac579-82ed-40da-bd6d-3d893df12acf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:54.820893 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1117/2243] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:27:54.824868 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0499c48-cf40-4b53-b629-b30fc73f7a34 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:54.834840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0499c48-cf40-4b53-b629-b30fc73f7a34 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:54.839436 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b195fb3-4cb1-4770-97cc-5ef3f4799d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:54.840012 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b195fb3-4cb1-4770-97cc-5ef3f4799d57 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:54.840287 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b195fb3-4cb1-4770-97cc-5ef3f4799d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:54.840552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b195fb3-4cb1-4770-97cc-5ef3f4799d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:54.841180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b195fb3-4cb1-4770-97cc-5ef3f4799d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:54.842810 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b195fb3-4cb1-4770-97cc-5ef3f4799d57 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:54.854614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1127/2244] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:54.858603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aac579-82ed-40da-bd6d-3d893df12acf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:54.868624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aac579-82ed-40da-bd6d-3d893df12acf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:54.873874 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ede042-c229-407f-a286-6494a60e01fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:54.874570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ede042-c229-407f-a286-6494a60e01fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:54.874853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ede042-c229-407f-a286-6494a60e01fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:54.875133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ede042-c229-407f-a286-6494a60e01fe admin admin] PATH_INFO: `/v3/projects/d278bb6853c04c42825ecb0363080c33/users/6f78d951cbe34531ad5639393e3750f6/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:54.889547 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ede042-c229-407f-a286-6494a60e01fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d278bb6853c04c42825ecb0363080c33, user_id=6f78d951cbe34531ad5639393e3750f6, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:54.892261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ede042-c229-407f-a286-6494a60e01fe admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:54.904811 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1118/2245] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:54 2022] PUT /identity/v3/projects/d278bb6853c04c42825ecb0363080c33/users/6f78d951cbe34531ad5639393e3750f6/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:54.914971 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aa3a0-569d-4616-a4e0-71e27c120752 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:54.915256 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aa3a0-569d-4616-a4e0-71e27c120752 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:54.915544 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aa3a0-569d-4616-a4e0-71e27c120752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:54.943979 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-e59aa3a0-569d-4616-a4e0-71e27c120752 None None] MFA Rules not processed for user `6f78d951cbe34531ad5639393e3750f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:54.945276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e59aa3a0-569d-4616-a4e0-71e27c120752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:55.010162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1128/2246] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:55.014755 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ede042-c229-407f-a286-6494a60e01fe admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:55.080175 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ede042-c229-407f-a286-6494a60e01fe admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:55.085231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5273b-2d6b-4c77-94a9-14a1150f4e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:55.085894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5273b-2d6b-4c77-94a9-14a1150f4e29 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:55.086168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5273b-2d6b-4c77-94a9-14a1150f4e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:55.086538 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5273b-2d6b-4c77-94a9-14a1150f4e29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:55.087167 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5273b-2d6b-4c77-94a9-14a1150f4e29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:55.089052 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5273b-2d6b-4c77-94a9-14a1150f4e29 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:55.108315 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccdbbc-a5c1-434c-9c5b-f5b71d7b4cd2 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:55.108601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccdbbc-a5c1-434c-9c5b-f5b71d7b4cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:55.108919 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccdbbc-a5c1-434c-9c5b-f5b71d7b4cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:55.119757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1119/2247] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:55 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:55.123990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5273b-2d6b-4c77-94a9-14a1150f4e29 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:55.128838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ceccdbbc-a5c1-434c-9c5b-f5b71d7b4cd2 None None] MFA Rules not processed for user `ef2ca1a30a76468cbe02b20efd6ca37e`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:55.130059 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ceccdbbc-a5c1-434c-9c5b-f5b71d7b4cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:55.154812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:55.187067 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1129/2248] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:27:55.201203 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5273b-2d6b-4c77-94a9-14a1150f4e29 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:55.206274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6216e-ea3c-468b-96eb-c19b23026119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:55.206900 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6216e-ea3c-468b-96eb-c19b23026119 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:55.207275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6216e-ea3c-468b-96eb-c19b23026119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:55.207436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6216e-ea3c-468b-96eb-c19b23026119 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:55.208103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6216e-ea3c-468b-96eb-c19b23026119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:55.209988 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6216e-ea3c-468b-96eb-c19b23026119 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:55.232518 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1120/2249] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:55 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:55.237005 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceccdbbc-a5c1-434c-9c5b-f5b71d7b4cd2 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:55.249744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceccdbbc-a5c1-434c-9c5b-f5b71d7b4cd2 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:55.254324 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643db8d1-392f-465a-8578-947bf41f4a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:55.254918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643db8d1-392f-465a-8578-947bf41f4a9b admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:55.255197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643db8d1-392f-465a-8578-947bf41f4a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:55.255466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643db8d1-392f-465a-8578-947bf41f4a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:55.256290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643db8d1-392f-465a-8578-947bf41f4a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:55.257875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643db8d1-392f-465a-8578-947bf41f4a9b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:55.268611 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1130/2250] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:55.272724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6216e-ea3c-468b-96eb-c19b23026119 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:55.283665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6216e-ea3c-468b-96eb-c19b23026119 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:55.288396 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515535f-8ff7-4053-9545-78c54d81a105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:55.288992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515535f-8ff7-4053-9545-78c54d81a105 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:55.289272 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515535f-8ff7-4053-9545-78c54d81a105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:55.289542 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515535f-8ff7-4053-9545-78c54d81a105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:55.290179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515535f-8ff7-4053-9545-78c54d81a105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:55.291816 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515535f-8ff7-4053-9545-78c54d81a105 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:55.303049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1121/2251] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:27:55.306509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643db8d1-392f-465a-8578-947bf41f4a9b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:55.320431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643db8d1-392f-465a-8578-947bf41f4a9b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:55.325506 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fcea83-56a1-4729-b823-6fba048f8a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:55.326172 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fcea83-56a1-4729-b823-6fba048f8a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:55.326638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fcea83-56a1-4729-b823-6fba048f8a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:55.326936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fcea83-56a1-4729-b823-6fba048f8a8d admin admin] PATH_INFO: `/v3/projects/7f5c41b16ce3484d9ce4f6aed1ac90e1/users/66a98072825141c8a104e4e375172480/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:55.343901 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fcea83-56a1-4729-b823-6fba048f8a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f5c41b16ce3484d9ce4f6aed1ac90e1, user_id=66a98072825141c8a104e4e375172480, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:55.347992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fcea83-56a1-4729-b823-6fba048f8a8d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:55.363895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1131/2252] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:55 2022] PUT /identity/v3/projects/7f5c41b16ce3484d9ce4f6aed1ac90e1/users/66a98072825141c8a104e4e375172480/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:55.375442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a3f5c-6004-4418-bea2-50835d4f2ae6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:55.375965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a3f5c-6004-4418-bea2-50835d4f2ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:55.375965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a3f5c-6004-4418-bea2-50835d4f2ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:55.411042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d05a3f5c-6004-4418-bea2-50835d4f2ae6 None None] MFA Rules not processed for user `66a98072825141c8a104e4e375172480`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:55.412634 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d05a3f5c-6004-4418-bea2-50835d4f2ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:55.492018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1122/2253] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:27:55.493803 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fcea83-56a1-4729-b823-6fba048f8a8d admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:55.540687 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:55.562206 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fcea83-56a1-4729-b823-6fba048f8a8d admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:55.566976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a0ec5-7544-493c-a6e5-669f084cbbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:55.567582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a0ec5-7544-493c-a6e5-669f084cbbca admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:55.567865 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a0ec5-7544-493c-a6e5-669f084cbbca admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:55.568262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a0ec5-7544-493c-a6e5-669f084cbbca admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:55.568903 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a0ec5-7544-493c-a6e5-669f084cbbca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:55.570866 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a0ec5-7544-493c-a6e5-669f084cbbca admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:55.606699 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1132/2254] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:55 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:55.608238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a3f5c-6004-4418-bea2-50835d4f2ae6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:55.621298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a3f5c-6004-4418-bea2-50835d4f2ae6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:55.681564 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb1a8a-db43-490b-9a79-9cf6816366b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:55.682159 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb1a8a-db43-490b-9a79-9cf6816366b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:55.682470 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb1a8a-db43-490b-9a79-9cf6816366b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:55.682739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb1a8a-db43-490b-9a79-9cf6816366b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:55.683407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb1a8a-db43-490b-9a79-9cf6816366b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:55.685333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb1a8a-db43-490b-9a79-9cf6816366b5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:55.708894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1123/2255] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:55 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:55.713466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a0ec5-7544-493c-a6e5-669f084cbbca admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:55.728568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a0ec5-7544-493c-a6e5-669f084cbbca admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:55.751846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746a69c-ff2d-45be-8ccb-8604cb261e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:55.752843 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746a69c-ff2d-45be-8ccb-8604cb261e5e admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:55.753296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746a69c-ff2d-45be-8ccb-8604cb261e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:55.753760 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746a69c-ff2d-45be-8ccb-8604cb261e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:55.754595 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746a69c-ff2d-45be-8ccb-8604cb261e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:55.759601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746a69c-ff2d-45be-8ccb-8604cb261e5e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:55.775074 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1133/2256] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:55.779713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb1a8a-db43-490b-9a79-9cf6816366b5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:55.796082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb1a8a-db43-490b-9a79-9cf6816366b5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:55.801900 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546bbf10-a99c-45ba-8cb2-787ce7fdabdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:55.803095 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bbf10-a99c-45ba-8cb2-787ce7fdabdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:55.803595 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bbf10-a99c-45ba-8cb2-787ce7fdabdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:55.804203 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bbf10-a99c-45ba-8cb2-787ce7fdabdc admin admin] PATH_INFO: `/v3/projects/760d3d09aec2449f86c573b78a802c5f/users/14d52955b18041a5bd0b2fed5ca90ba1/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:55.823752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bbf10-a99c-45ba-8cb2-787ce7fdabdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=760d3d09aec2449f86c573b78a802c5f, user_id=14d52955b18041a5bd0b2fed5ca90ba1, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:55.828193 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bbf10-a99c-45ba-8cb2-787ce7fdabdc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:55.859035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1124/2257] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:55 2022] PUT /identity/v3/projects/760d3d09aec2449f86c573b78a802c5f/users/14d52955b18041a5bd0b2fed5ca90ba1/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:55.864355 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746a69c-ff2d-45be-8ccb-8604cb261e5e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:55.877984 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746a69c-ff2d-45be-8ccb-8604cb261e5e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:55.946634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ba397-44bc-4c4f-83ce-287d7e416ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:55.947242 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ba397-44bc-4c4f-83ce-287d7e416ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:55.947519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ba397-44bc-4c4f-83ce-287d7e416ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:55.947787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ba397-44bc-4c4f-83ce-287d7e416ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:55.948431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ba397-44bc-4c4f-83ce-287d7e416ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:55.950778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ba397-44bc-4c4f-83ce-287d7e416ba3 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:55.960630 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1134/2258] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:55.964853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546bbf10-a99c-45ba-8cb2-787ce7fdabdc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:55.970832 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:55.976620 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546bbf10-a99c-45ba-8cb2-787ce7fdabdc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:55.982575 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c08640-6707-4610-a5d3-77c940cd3e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:55.983350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c08640-6707-4610-a5d3-77c940cd3e95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:55.983892 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c08640-6707-4610-a5d3-77c940cd3e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:55.984263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c08640-6707-4610-a5d3-77c940cd3e95 admin admin] PATH_INFO: `/v3/projects/760d3d09aec2449f86c573b78a802c5f/users/14d52955b18041a5bd0b2fed5ca90ba1/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:55.986369 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c08640-6707-4610-a5d3-77c940cd3e95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=760d3d09aec2449f86c573b78a802c5f, user_id=14d52955b18041a5bd0b2fed5ca90ba1, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:55.989015 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c08640-6707-4610-a5d3-77c940cd3e95 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:56.000539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1125/2259] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:55 2022] PUT /identity/v3/projects/760d3d09aec2449f86c573b78a802c5f/users/14d52955b18041a5bd0b2fed5ca90ba1/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:56.010556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09267387-49d5-4ccf-94d2-abc0cfb887db None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:56.010847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09267387-49d5-4ccf-94d2-abc0cfb887db None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:56.011132 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09267387-49d5-4ccf-94d2-abc0cfb887db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:56.040701 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-09267387-49d5-4ccf-94d2-abc0cfb887db None None] MFA Rules not processed for user `14d52955b18041a5bd0b2fed5ca90ba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:56.041976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-09267387-49d5-4ccf-94d2-abc0cfb887db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:56.122232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1135/2260] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:27:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:56.134909 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c08640-6707-4610-a5d3-77c940cd3e95 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:56.137857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-01c08640-6707-4610-a5d3-77c940cd3e95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:56.206569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c08640-6707-4610-a5d3-77c940cd3e95 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:56.210233 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497af46-6551-4143-bd6c-9370df057312 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:56.210891 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497af46-6551-4143-bd6c-9370df057312 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:56.211164 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497af46-6551-4143-bd6c-9370df057312 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:56.211429 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497af46-6551-4143-bd6c-9370df057312 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:56.267392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1497af46-6551-4143-bd6c-9370df057312 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:56.268826 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1497af46-6551-4143-bd6c-9370df057312 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:56.277599 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1126/2261] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:57.820128 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09267387-49d5-4ccf-94d2-abc0cfb887db None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:57.830015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09267387-49d5-4ccf-94d2-abc0cfb887db None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:57.833882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7548e4-a5f2-4f89-960c-222547cd1958 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:57.834299 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7548e4-a5f2-4f89-960c-222547cd1958 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:57.834581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7548e4-a5f2-4f89-960c-222547cd1958 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:57.834854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7548e4-a5f2-4f89-960c-222547cd1958 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:57.866618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:57.894322 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7548e4-a5f2-4f89-960c-222547cd1958 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:57.895768 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7548e4-a5f2-4f89-960c-222547cd1958 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:57.905074 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1136/2262] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:58.500817 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497af46-6551-4143-bd6c-9370df057312 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:58.509394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497af46-6551-4143-bd6c-9370df057312 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:58.513497 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7260b-c2cd-413a-9ba6-2990a3622279 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:58.514085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7260b-c2cd-413a-9ba6-2990a3622279 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:58.514400 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7260b-c2cd-413a-9ba6-2990a3622279 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:58.514668 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7260b-c2cd-413a-9ba6-2990a3622279 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:58.523893 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7260b-c2cd-413a-9ba6-2990a3622279 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:58.525322 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7260b-c2cd-413a-9ba6-2990a3622279 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:58.535352 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1127/2263] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:02.401817 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7548e4-a5f2-4f89-960c-222547cd1958 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:02.473237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7548e4-a5f2-4f89-960c-222547cd1958 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:02.478852 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95707b0-267f-4335-8a7d-a04226f75528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:02.479721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95707b0-267f-4335-8a7d-a04226f75528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:02.480509 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95707b0-267f-4335-8a7d-a04226f75528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:02.480785 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95707b0-267f-4335-8a7d-a04226f75528 admin admin] PATH_INFO: `/v3/users/6f78d951cbe34531ad5639393e3750f6` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:02.482127 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95707b0-267f-4335-8a7d-a04226f75528 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f78d951cbe34531ad5639393e3750f6)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:02.484992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95707b0-267f-4335-8a7d-a04226f75528 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:02.545810 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f95707b0-267f-4335-8a7d-a04226f75528 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f78d951cbe34531ad5639393e3750f6'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:02.552951 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f95707b0-267f-4335-8a7d-a04226f75528 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f78d951cbe34531ad5639393e3750f6'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:02.561108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f95707b0-267f-4335-8a7d-a04226f75528 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f78d951cbe34531ad5639393e3750f6'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:02.561725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f95707b0-267f-4335-8a7d-a04226f75528 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f78d951cbe34531ad5639393e3750f6'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:02.562372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f95707b0-267f-4335-8a7d-a04226f75528 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f78d951cbe34531ad5639393e3750f6'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:02.581109 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1137/2264] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:02 2022] DELETE /identity/v3/users/6f78d951cbe34531ad5639393e3750f6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:02.585836 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dcb48-cee7-4aca-a11e-5aae59d03993 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:02.586345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dcb48-cee7-4aca-a11e-5aae59d03993 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:02.586498 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dcb48-cee7-4aca-a11e-5aae59d03993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:02.607912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-566dcb48-cee7-4aca-a11e-5aae59d03993 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:02.609183 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-566dcb48-cee7-4aca-a11e-5aae59d03993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:02.691062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1128/2265] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:02.703416 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95707b0-267f-4335-8a7d-a04226f75528 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:02.705530 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f95707b0-267f-4335-8a7d-a04226f75528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:02.780288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95707b0-267f-4335-8a7d-a04226f75528 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:02.784558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364d821-3223-4a81-9b9b-e64cc16655de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:02.785318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364d821-3223-4a81-9b9b-e64cc16655de service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:02.785662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364d821-3223-4a81-9b9b-e64cc16655de service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:02.785999 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364d821-3223-4a81-9b9b-e64cc16655de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:02.795188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:02.812436 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364d821-3223-4a81-9b9b-e64cc16655de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:02.814087 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364d821-3223-4a81-9b9b-e64cc16655de service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:02.826723 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1138/2266] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:28:03.135526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dcb48-cee7-4aca-a11e-5aae59d03993 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:03.137916 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-566dcb48-cee7-4aca-a11e-5aae59d03993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:03.153168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dcb48-cee7-4aca-a11e-5aae59d03993 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:03.159163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae94f0-7404-4230-b0dc-8bdf96a581a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:03.160192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae94f0-7404-4230-b0dc-8bdf96a581a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:03.160573 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae94f0-7404-4230-b0dc-8bdf96a581a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:03.160946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae94f0-7404-4230-b0dc-8bdf96a581a3 admin admin] PATH_INFO: `/v3/projects/d278bb6853c04c42825ecb0363080c33` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:03.162432 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae94f0-7404-4230-b0dc-8bdf96a581a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d278bb6853c04c42825ecb0363080c33)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:03.165391 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae94f0-7404-4230-b0dc-8bdf96a581a3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:03.222153 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:03.252941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f5ae94f0-7404-4230-b0dc-8bdf96a581a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd278bb6853c04c42825ecb0363080c33', 'request_id': 'req-f5ae94f0-7404-4230-b0dc-8bdf96a581a3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.263958 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f5ae94f0-7404-4230-b0dc-8bdf96a581a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd278bb6853c04c42825ecb0363080c33', 'request_id': 'req-f5ae94f0-7404-4230-b0dc-8bdf96a581a3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.276286 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f5ae94f0-7404-4230-b0dc-8bdf96a581a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd278bb6853c04c42825ecb0363080c33', 'request_id': 'req-f5ae94f0-7404-4230-b0dc-8bdf96a581a3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.295656 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f5ae94f0-7404-4230-b0dc-8bdf96a581a3 admin admin] The token cache is being invalidate because project d278bb6853c04c42825ecb0363080c33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:03.296022 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f5ae94f0-7404-4230-b0dc-8bdf96a581a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.296759 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f5ae94f0-7404-4230-b0dc-8bdf96a581a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.299552 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1129/2267] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:03 2022] DELETE /identity/v3/projects/d278bb6853c04c42825ecb0363080c33 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:03.304147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364d821-3223-4a81-9b9b-e64cc16655de service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:03.317498 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364d821-3223-4a81-9b9b-e64cc16655de service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:03.385280 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfbe4d-3278-40e9-abb4-51dbd7e3534a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:03.386108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfbe4d-3278-40e9-abb4-51dbd7e3534a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:03.387083 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfbe4d-3278-40e9-abb4-51dbd7e3534a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:03.387364 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfbe4d-3278-40e9-abb4-51dbd7e3534a admin admin] PATH_INFO: `/v3/users/66a98072825141c8a104e4e375172480` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:03.388505 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bfbe4d-3278-40e9-abb4-51dbd7e3534a admin admin] RBAC: Authorizing `identity:delete_user(user_id=66a98072825141c8a104e4e375172480)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:03.396022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bfbe4d-3278-40e9-abb4-51dbd7e3534a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:03.452915 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b6bfbe4d-3278-40e9-abb4-51dbd7e3534a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66a98072825141c8a104e4e375172480'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.461922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b6bfbe4d-3278-40e9-abb4-51dbd7e3534a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66a98072825141c8a104e4e375172480'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.471719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b6bfbe4d-3278-40e9-abb4-51dbd7e3534a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66a98072825141c8a104e4e375172480'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.472331 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b6bfbe4d-3278-40e9-abb4-51dbd7e3534a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66a98072825141c8a104e4e375172480'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.472937 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b6bfbe4d-3278-40e9-abb4-51dbd7e3534a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66a98072825141c8a104e4e375172480'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.490429 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1139/2268] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:03 2022] DELETE /identity/v3/users/66a98072825141c8a104e4e375172480 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:03.812648 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae94f0-7404-4230-b0dc-8bdf96a581a3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:03.815714 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f5ae94f0-7404-4230-b0dc-8bdf96a581a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:03.845307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae94f0-7404-4230-b0dc-8bdf96a581a3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:03.924249 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8c632-3db8-4428-9505-d75a64686d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:03.925592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8c632-3db8-4428-9505-d75a64686d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:03.926095 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8c632-3db8-4428-9505-d75a64686d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:03.926686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8c632-3db8-4428-9505-d75a64686d48 admin admin] PATH_INFO: `/v3/projects/7f5c41b16ce3484d9ce4f6aed1ac90e1` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:03.930530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8c632-3db8-4428-9505-d75a64686d48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f5c41b16ce3484d9ce4f6aed1ac90e1)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:03.939162 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8c632-3db8-4428-9505-d75a64686d48 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:04.023229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bba8c632-3db8-4428-9505-d75a64686d48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f5c41b16ce3484d9ce4f6aed1ac90e1', 'request_id': 'req-bba8c632-3db8-4428-9505-d75a64686d48'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:04.030236 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bba8c632-3db8-4428-9505-d75a64686d48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f5c41b16ce3484d9ce4f6aed1ac90e1', 'request_id': 'req-bba8c632-3db8-4428-9505-d75a64686d48'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:04.041086 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bba8c632-3db8-4428-9505-d75a64686d48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f5c41b16ce3484d9ce4f6aed1ac90e1', 'request_id': 'req-bba8c632-3db8-4428-9505-d75a64686d48'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:04.054685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bba8c632-3db8-4428-9505-d75a64686d48 admin admin] The token cache is being invalidate because project 7f5c41b16ce3484d9ce4f6aed1ac90e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:04.055024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bba8c632-3db8-4428-9505-d75a64686d48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:04.056206 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bba8c632-3db8-4428-9505-d75a64686d48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:04.057734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1130/2269] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:03 2022] DELETE /identity/v3/projects/7f5c41b16ce3484d9ce4f6aed1ac90e1 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:04.063156 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfbe4d-3278-40e9-abb4-51dbd7e3534a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:04.065492 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b6bfbe4d-3278-40e9-abb4-51dbd7e3534a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:04.090494 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfbe4d-3278-40e9-abb4-51dbd7e3534a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:04.157860 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b07224-ebd5-43df-b608-660d6b5c6369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:04.158610 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b07224-ebd5-43df-b608-660d6b5c6369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:04.158961 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b07224-ebd5-43df-b608-660d6b5c6369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:04.159327 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b07224-ebd5-43df-b608-660d6b5c6369 admin admin] PATH_INFO: `/v3/users/14d52955b18041a5bd0b2fed5ca90ba1` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:04.160595 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b07224-ebd5-43df-b608-660d6b5c6369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14d52955b18041a5bd0b2fed5ca90ba1)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:04.162719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b07224-ebd5-43df-b608-660d6b5c6369 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:04.193960 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:04.235737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-94b07224-ebd5-43df-b608-660d6b5c6369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14d52955b18041a5bd0b2fed5ca90ba1'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:04.245586 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-94b07224-ebd5-43df-b608-660d6b5c6369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14d52955b18041a5bd0b2fed5ca90ba1'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:04.253889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-94b07224-ebd5-43df-b608-660d6b5c6369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14d52955b18041a5bd0b2fed5ca90ba1'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:04.254560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-94b07224-ebd5-43df-b608-660d6b5c6369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14d52955b18041a5bd0b2fed5ca90ba1'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:04.255176 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-94b07224-ebd5-43df-b608-660d6b5c6369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14d52955b18041a5bd0b2fed5ca90ba1'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:04.281845 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1140/2270] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:04 2022] DELETE /identity/v3/users/14d52955b18041a5bd0b2fed5ca90ba1 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:04.679246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8c632-3db8-4428-9505-d75a64686d48 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:04.680729 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bba8c632-3db8-4428-9505-d75a64686d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:04.719450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8c632-3db8-4428-9505-d75a64686d48 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:04.783826 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:04.815023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c37a13-8f97-4f05-afe3-289b704b95ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:04.815812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c37a13-8f97-4f05-afe3-289b704b95ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:04.816760 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c37a13-8f97-4f05-afe3-289b704b95ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:04.817122 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c37a13-8f97-4f05-afe3-289b704b95ed admin admin] PATH_INFO: `/v3/projects/760d3d09aec2449f86c573b78a802c5f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:04.818469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c37a13-8f97-4f05-afe3-289b704b95ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=760d3d09aec2449f86c573b78a802c5f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:04.820494 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c37a13-8f97-4f05-afe3-289b704b95ed admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:04.870015 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-89c37a13-8f97-4f05-afe3-289b704b95ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '760d3d09aec2449f86c573b78a802c5f', 'request_id': 'req-89c37a13-8f97-4f05-afe3-289b704b95ed'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:04.876888 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-89c37a13-8f97-4f05-afe3-289b704b95ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '760d3d09aec2449f86c573b78a802c5f', 'request_id': 'req-89c37a13-8f97-4f05-afe3-289b704b95ed'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:04.884005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-89c37a13-8f97-4f05-afe3-289b704b95ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '760d3d09aec2449f86c573b78a802c5f', 'request_id': 'req-89c37a13-8f97-4f05-afe3-289b704b95ed'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:04.895947 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-89c37a13-8f97-4f05-afe3-289b704b95ed admin admin] The token cache is being invalidate because project 760d3d09aec2449f86c573b78a802c5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:04.896238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-89c37a13-8f97-4f05-afe3-289b704b95ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:04.896878 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-89c37a13-8f97-4f05-afe3-289b704b95ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:04.898395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1131/2271] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:04 2022] DELETE /identity/v3/projects/760d3d09aec2449f86c573b78a802c5f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:04.903897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0657c6b-2f3f-425b-a716-86f4b4360e5e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:04.904426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0657c6b-2f3f-425b-a716-86f4b4360e5e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:04.904426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0657c6b-2f3f-425b-a716-86f4b4360e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:04.927065 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-e0657c6b-2f3f-425b-a716-86f4b4360e5e None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:04.928297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e0657c6b-2f3f-425b-a716-86f4b4360e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:04.999846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1141/2272] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:28:05.008748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b38d7e-514e-4343-ba2f-669d4dd288eb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:05.009035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b38d7e-514e-4343-ba2f-669d4dd288eb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:05.009311 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b38d7e-514e-4343-ba2f-669d4dd288eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:05.029372 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-28b38d7e-514e-4343-ba2f-669d4dd288eb None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:05.030667 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-28b38d7e-514e-4343-ba2f-669d4dd288eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:05.039625 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1132/2273] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:05.044719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0657c6b-2f3f-425b-a716-86f4b4360e5e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:05.054788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0657c6b-2f3f-425b-a716-86f4b4360e5e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:05.060367 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fdfe43-2386-44a3-ab56-659fbd6575c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:05.061091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fdfe43-2386-44a3-ab56-659fbd6575c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:05.061366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fdfe43-2386-44a3-ab56-659fbd6575c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:05.061636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fdfe43-2386-44a3-ab56-659fbd6575c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:05.062358 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fdfe43-2386-44a3-ab56-659fbd6575c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:28:05.063282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fdfe43-2386-44a3-ab56-659fbd6575c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:05.065223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fdfe43-2386-44a3-ab56-659fbd6575c7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:05.082224 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1142/2274] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:28:05.083367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b38d7e-514e-4343-ba2f-669d4dd288eb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:05.092927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b38d7e-514e-4343-ba2f-669d4dd288eb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:05.099094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799d438-7fe5-46fe-b140-5924d7cfa880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:05.099751 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799d438-7fe5-46fe-b140-5924d7cfa880 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:05.100311 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799d438-7fe5-46fe-b140-5924d7cfa880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:05.100575 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799d438-7fe5-46fe-b140-5924d7cfa880 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:05.101212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799d438-7fe5-46fe-b140-5924d7cfa880 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:05.103171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799d438-7fe5-46fe-b140-5924d7cfa880 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:05.130577 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1133/2275] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:28:05 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:05.134905 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fdfe43-2386-44a3-ab56-659fbd6575c7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:05.146048 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fdfe43-2386-44a3-ab56-659fbd6575c7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:05.215158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c051d-8123-4d70-b50a-0f283195ffc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:05.216081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c051d-8123-4d70-b50a-0f283195ffc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:05.216513 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c051d-8123-4d70-b50a-0f283195ffc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:05.217047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c051d-8123-4d70-b50a-0f283195ffc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:05.217994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483c051d-8123-4d70-b50a-0f283195ffc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:05.220249 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483c051d-8123-4d70-b50a-0f283195ffc3 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:05.253934 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1143/2276] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:28:05 2022] POST /identity/v3/users => generated 285 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:28:05.255251 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799d438-7fe5-46fe-b140-5924d7cfa880 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:05.265768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799d438-7fe5-46fe-b140-5924d7cfa880 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:05.270541 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2834c5e-1a0e-4b2a-8af9-fa1ba9849883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:05.271140 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2834c5e-1a0e-4b2a-8af9-fa1ba9849883 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:05.271420 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2834c5e-1a0e-4b2a-8af9-fa1ba9849883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:05.271699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2834c5e-1a0e-4b2a-8af9-fa1ba9849883 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:05.272349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2834c5e-1a0e-4b2a-8af9-fa1ba9849883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:05.274082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2834c5e-1a0e-4b2a-8af9-fa1ba9849883 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:05.284369 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1134/2277] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:05.288169 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c051d-8123-4d70-b50a-0f283195ffc3 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:05.299411 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c051d-8123-4d70-b50a-0f283195ffc3 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:05.303626 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7fc30f-205e-4785-8eec-f4ce2dc2cc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:05.304240 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fc30f-205e-4785-8eec-f4ce2dc2cc1c admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:05.304507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fc30f-205e-4785-8eec-f4ce2dc2cc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:05.304781 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fc30f-205e-4785-8eec-f4ce2dc2cc1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:05.305433 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7fc30f-205e-4785-8eec-f4ce2dc2cc1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:05.307144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7fc30f-205e-4785-8eec-f4ce2dc2cc1c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:05.309916 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:05.323376 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1144/2278] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:05.324798 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2834c5e-1a0e-4b2a-8af9-fa1ba9849883 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:05.335114 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2834c5e-1a0e-4b2a-8af9-fa1ba9849883 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:05.340136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df8dee-afd7-40fb-b6fc-480278f9e36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:05.340783 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df8dee-afd7-40fb-b6fc-480278f9e36f admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:05.341057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df8dee-afd7-40fb-b6fc-480278f9e36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:05.341322 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df8dee-afd7-40fb-b6fc-480278f9e36f admin admin] PATH_INFO: `/v3/projects/e7edd39524494605ae420bee713126e4/users/97ad561b75214de8bfdf7cde1508d752/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:05.357258 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df8dee-afd7-40fb-b6fc-480278f9e36f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7edd39524494605ae420bee713126e4, user_id=97ad561b75214de8bfdf7cde1508d752, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:05.360059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df8dee-afd7-40fb-b6fc-480278f9e36f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:05.373832 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1135/2279] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:05 2022] PUT /identity/v3/projects/e7edd39524494605ae420bee713126e4/users/97ad561b75214de8bfdf7cde1508d752/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:05.384468 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b4dde-8267-4edd-a85b-2ccca487808c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:05.385004 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b4dde-8267-4edd-a85b-2ccca487808c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:05.385288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b4dde-8267-4edd-a85b-2ccca487808c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:05.413966 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-821b4dde-8267-4edd-a85b-2ccca487808c None None] MFA Rules not processed for user `97ad561b75214de8bfdf7cde1508d752`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:05.415283 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-821b4dde-8267-4edd-a85b-2ccca487808c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:05.492693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1145/2280] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:28:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:05.504237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df8dee-afd7-40fb-b6fc-480278f9e36f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:05.506034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-74df8dee-afd7-40fb-b6fc-480278f9e36f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:05.587560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df8dee-afd7-40fb-b6fc-480278f9e36f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:05.591773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be67e2-cf2c-4667-a70c-c5021ba3b0a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7776a6a1faf34b369e399d83bb3e18c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:05.592371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be67e2-cf2c-4667-a70c-c5021ba3b0a9 service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:05.592645 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be67e2-cf2c-4667-a70c-c5021ba3b0a9 service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:05.592909 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be67e2-cf2c-4667-a70c-c5021ba3b0a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:05.598151 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:05.601978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be67e2-cf2c-4667-a70c-c5021ba3b0a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:05.604072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be67e2-cf2c-4667-a70c-c5021ba3b0a9 service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:05.615911 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1136/2281] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:05.966874 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be7543-ebf8-4715-a38b-15397adc926e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:05.967374 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be7543-ebf8-4715-a38b-15397adc926e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:05.967492 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be7543-ebf8-4715-a38b-15397adc926e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:05.988006 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-16be7543-ebf8-4715-a38b-15397adc926e None None] MFA Rules not processed for user `1fd1b40cbd884915bdfac150f9718e32`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:05.989217 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-16be7543-ebf8-4715-a38b-15397adc926e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:06.049962 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1146/2282] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:28:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:10.528423 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd07cc6-afe6-4264-a862-5736696118fa None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:10.529005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd07cc6-afe6-4264-a862-5736696118fa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:10.529169 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd07cc6-afe6-4264-a862-5736696118fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:10.548981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4bd07cc6-afe6-4264-a862-5736696118fa None None] MFA Rules not processed for user `b88ccd3e5c4c48d3a27f4eaa7874e28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:10.550174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4bd07cc6-afe6-4264-a862-5736696118fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:10.607576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1137/2283] 10.222.0.39 () {58 vars in 1009 bytes} [Mon Aug 8 02:28:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:10.619455 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be7543-ebf8-4715-a38b-15397adc926e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:10.620857 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-16be7543-ebf8-4715-a38b-15397adc926e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:10.692843 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be7543-ebf8-4715-a38b-15397adc926e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:10.697837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3f4f2-d111-4354-a957-0c866faf0d95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac60b4d621234225830c94edfac400e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:10.698470 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3f4f2-d111-4354-a957-0c866faf0d95 service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:10.698750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3f4f2-d111-4354-a957-0c866faf0d95 service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:10.699017 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3f4f2-d111-4354-a957-0c866faf0d95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:10.707845 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3f4f2-d111-4354-a957-0c866faf0d95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:10.709436 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3f4f2-d111-4354-a957-0c866faf0d95 service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:10.718381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1147/2284] 10.222.0.39 () {60 vars in 1384 bytes} [Mon Aug 8 02:28:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:28:20.600278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=28 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:20.714758 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:34.298153 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76dbd9-a6b6-4019-bc14-eb7135388bac None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:34.298852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76dbd9-a6b6-4019-bc14-eb7135388bac None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:34.298852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76dbd9-a6b6-4019-bc14-eb7135388bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:34.320335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-dc76dbd9-a6b6-4019-bc14-eb7135388bac None None] MFA Rules not processed for user `b88ccd3e5c4c48d3a27f4eaa7874e28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:34.321541 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dc76dbd9-a6b6-4019-bc14-eb7135388bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:34.329270 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1138/2285] 10.222.0.39 () {58 vars in 967 bytes} [Mon Aug 8 02:28:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:34.341333 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3f4f2-d111-4354-a957-0c866faf0d95 service swift] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:34.352005 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3f4f2-d111-4354-a957-0c866faf0d95 service swift] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:34.357479 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63aca2-743d-4d22-895c-98b863aa1ad1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac60b4d621234225830c94edfac400e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:34.358061 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63aca2-743d-4d22-895c-98b863aa1ad1 service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:34.358397 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63aca2-743d-4d22-895c-98b863aa1ad1 service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:34.358669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63aca2-743d-4d22-895c-98b863aa1ad1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:34.366729 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63aca2-743d-4d22-895c-98b863aa1ad1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:34.368278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63aca2-743d-4d22-895c-98b863aa1ad1 service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:34.376730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1148/2286] 10.222.0.39 () {60 vars in 1384 bytes} [Mon Aug 8 02:28:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 02:28:36.446829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76dbd9-a6b6-4019-bc14-eb7135388bac None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:36.450543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dc76dbd9-a6b6-4019-bc14-eb7135388bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:36.533958 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76dbd9-a6b6-4019-bc14-eb7135388bac None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:36.538645 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e723d8-7977-4c39-ba96-8ac7e6fa6e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:36.539272 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e723d8-7977-4c39-ba96-8ac7e6fa6e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:36.539556 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e723d8-7977-4c39-ba96-8ac7e6fa6e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:36.539845 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e723d8-7977-4c39-ba96-8ac7e6fa6e7c admin admin] PATH_INFO: `/v3/users/6916c2afef12490f921d99c5047b4dce` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:36.540967 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e723d8-7977-4c39-ba96-8ac7e6fa6e7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6916c2afef12490f921d99c5047b4dce)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:36.542931 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e723d8-7977-4c39-ba96-8ac7e6fa6e7c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:36.596970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-04e723d8-7977-4c39-ba96-8ac7e6fa6e7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6916c2afef12490f921d99c5047b4dce'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:36.604448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-04e723d8-7977-4c39-ba96-8ac7e6fa6e7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6916c2afef12490f921d99c5047b4dce'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:36.614443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-04e723d8-7977-4c39-ba96-8ac7e6fa6e7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6916c2afef12490f921d99c5047b4dce'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:36.615193 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-04e723d8-7977-4c39-ba96-8ac7e6fa6e7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6916c2afef12490f921d99c5047b4dce'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:36.615885 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-04e723d8-7977-4c39-ba96-8ac7e6fa6e7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6916c2afef12490f921d99c5047b4dce'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:36.642747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1139/2287] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:36 2022] DELETE /identity/v3/users/6916c2afef12490f921d99c5047b4dce => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:36.845097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63aca2-743d-4d22-895c-98b863aa1ad1 service swift] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:36.858571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63aca2-743d-4d22-895c-98b863aa1ad1 service swift] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:36.926279 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0fb8b-0465-43b0-b214-9c1ed3072fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:36.926906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0fb8b-0465-43b0-b214-9c1ed3072fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:36.927183 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0fb8b-0465-43b0-b214-9c1ed3072fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:36.927613 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0fb8b-0465-43b0-b214-9c1ed3072fb2 admin admin] PATH_INFO: `/v3/projects/d3cc8b8bc4964df4a494173af8540c45` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:36.928735 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0fb8b-0465-43b0-b214-9c1ed3072fb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3cc8b8bc4964df4a494173af8540c45)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:36.930870 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0fb8b-0465-43b0-b214-9c1ed3072fb2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:37.000894 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-25a0fb8b-0465-43b0-b214-9c1ed3072fb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3cc8b8bc4964df4a494173af8540c45', 'request_id': 'req-25a0fb8b-0465-43b0-b214-9c1ed3072fb2'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:37.006995 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-25a0fb8b-0465-43b0-b214-9c1ed3072fb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3cc8b8bc4964df4a494173af8540c45', 'request_id': 'req-25a0fb8b-0465-43b0-b214-9c1ed3072fb2'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:37.016145 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-25a0fb8b-0465-43b0-b214-9c1ed3072fb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3cc8b8bc4964df4a494173af8540c45', 'request_id': 'req-25a0fb8b-0465-43b0-b214-9c1ed3072fb2'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:37.027550 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-25a0fb8b-0465-43b0-b214-9c1ed3072fb2 admin admin] The token cache is being invalidate because project d3cc8b8bc4964df4a494173af8540c45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:37.027898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-25a0fb8b-0465-43b0-b214-9c1ed3072fb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:37.028568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-25a0fb8b-0465-43b0-b214-9c1ed3072fb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:37.030044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1149/2288] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:36 2022] DELETE /identity/v3/projects/d3cc8b8bc4964df4a494173af8540c45 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:37.035917 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fd84b-a7ff-4cfc-aa1d-abfa3c534ec6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:37.036201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fd84b-a7ff-4cfc-aa1d-abfa3c534ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:37.036478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fd84b-a7ff-4cfc-aa1d-abfa3c534ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:37.057455 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a46fd84b-a7ff-4cfc-aa1d-abfa3c534ec6 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:37.058716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a46fd84b-a7ff-4cfc-aa1d-abfa3c534ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:37.121075 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1140/2289] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:37.130060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e6247-384d-4eb6-9100-15a19d0ce2b4 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:37.130455 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e6247-384d-4eb6-9100-15a19d0ce2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:37.130796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e6247-384d-4eb6-9100-15a19d0ce2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:37.151393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-9b2e6247-384d-4eb6-9100-15a19d0ce2b4 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:37.152602 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9b2e6247-384d-4eb6-9100-15a19d0ce2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:37.160368 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1150/2290] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:37.164560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fd84b-a7ff-4cfc-aa1d-abfa3c534ec6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:37.174554 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fd84b-a7ff-4cfc-aa1d-abfa3c534ec6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:37.179674 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d98461-5889-4a43-bf60-5ebf4ee02440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:37.180345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d98461-5889-4a43-bf60-5ebf4ee02440 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:37.180693 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d98461-5889-4a43-bf60-5ebf4ee02440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:37.181035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d98461-5889-4a43-bf60-5ebf4ee02440 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:37.181754 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d98461-5889-4a43-bf60-5ebf4ee02440 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:28:37.182150 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d98461-5889-4a43-bf60-5ebf4ee02440 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:37.184404 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d98461-5889-4a43-bf60-5ebf4ee02440 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:37.193594 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:37.201353 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1141/2291] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:28:37.203099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e6247-384d-4eb6-9100-15a19d0ce2b4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:37.214618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e6247-384d-4eb6-9100-15a19d0ce2b4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:37.221834 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb77044-fd3a-4ff1-bb28-1cb518a233c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:37.222775 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb77044-fd3a-4ff1-bb28-1cb518a233c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:37.223144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb77044-fd3a-4ff1-bb28-1cb518a233c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:37.223491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb77044-fd3a-4ff1-bb28-1cb518a233c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:37.224332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb77044-fd3a-4ff1-bb28-1cb518a233c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:37.226440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb77044-fd3a-4ff1-bb28-1cb518a233c2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:37.259978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1151/2292] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:28:37 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:37.264608 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d98461-5889-4a43-bf60-5ebf4ee02440 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:37.275936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d98461-5889-4a43-bf60-5ebf4ee02440 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:37.337376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811d7d2-bc85-4f2f-830b-33181db63619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:37.337999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811d7d2-bc85-4f2f-830b-33181db63619 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:37.338308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811d7d2-bc85-4f2f-830b-33181db63619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:37.338584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811d7d2-bc85-4f2f-830b-33181db63619 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:37.339206 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811d7d2-bc85-4f2f-830b-33181db63619 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:37.341197 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811d7d2-bc85-4f2f-830b-33181db63619 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:37.363067 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1142/2293] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:28:37 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:37.367036 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb77044-fd3a-4ff1-bb28-1cb518a233c2 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:37.378641 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb77044-fd3a-4ff1-bb28-1cb518a233c2 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:37.383383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a564a38-5931-43e1-87df-22c525d49c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:37.383999 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a564a38-5931-43e1-87df-22c525d49c26 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:37.384278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a564a38-5931-43e1-87df-22c525d49c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:37.384548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a564a38-5931-43e1-87df-22c525d49c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:37.385196 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a564a38-5931-43e1-87df-22c525d49c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:37.386834 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a564a38-5931-43e1-87df-22c525d49c26 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:37.404906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1152/2294] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:37.408858 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811d7d2-bc85-4f2f-830b-33181db63619 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:37.420834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811d7d2-bc85-4f2f-830b-33181db63619 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:37.427143 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c3f19-4228-4be9-be67-54e085901edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:37.427756 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c3f19-4228-4be9-be67-54e085901edc admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:37.428036 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c3f19-4228-4be9-be67-54e085901edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:37.428336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c3f19-4228-4be9-be67-54e085901edc admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:37.428982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c3f19-4228-4be9-be67-54e085901edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:37.430657 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c3f19-4228-4be9-be67-54e085901edc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:37.445230 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1143/2295] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:37.446923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a564a38-5931-43e1-87df-22c525d49c26 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:37.458580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a564a38-5931-43e1-87df-22c525d49c26 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:37.464756 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54639472-e560-4eca-9bd1-ac2c93618fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:37.465538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54639472-e560-4eca-9bd1-ac2c93618fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:37.465825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54639472-e560-4eca-9bd1-ac2c93618fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:37.466118 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54639472-e560-4eca-9bd1-ac2c93618fbb admin admin] PATH_INFO: `/v3/projects/427fbcf122944aca83622b7fd0616982/users/eea82fd0321d4e88a7943681099aadd7/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:37.483039 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54639472-e560-4eca-9bd1-ac2c93618fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=427fbcf122944aca83622b7fd0616982, user_id=eea82fd0321d4e88a7943681099aadd7, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:37.485825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54639472-e560-4eca-9bd1-ac2c93618fbb admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:37.502121 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1153/2296] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:37 2022] PUT /identity/v3/projects/427fbcf122944aca83622b7fd0616982/users/eea82fd0321d4e88a7943681099aadd7/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:37.512484 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a072654-3ca9-4002-88ec-35824e2abf37 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:37.512984 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a072654-3ca9-4002-88ec-35824e2abf37 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:37.513087 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a072654-3ca9-4002-88ec-35824e2abf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:37.542018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4a072654-3ca9-4002-88ec-35824e2abf37 None None] MFA Rules not processed for user `eea82fd0321d4e88a7943681099aadd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:37.543853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4a072654-3ca9-4002-88ec-35824e2abf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:37.616050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1144/2297] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:28:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:37.628214 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54639472-e560-4eca-9bd1-ac2c93618fbb admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:37.630005 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-54639472-e560-4eca-9bd1-ac2c93618fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:37.645147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:37.712831 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54639472-e560-4eca-9bd1-ac2c93618fbb admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:37.719032 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce35609-067b-4a9e-b249-94faaf6ca293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:37.719672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce35609-067b-4a9e-b249-94faaf6ca293 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:37.719949 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce35609-067b-4a9e-b249-94faaf6ca293 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:37.720221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce35609-067b-4a9e-b249-94faaf6ca293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:37.729513 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce35609-067b-4a9e-b249-94faaf6ca293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:37.734857 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce35609-067b-4a9e-b249-94faaf6ca293 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:37.744754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1154/2298] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:47.612962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:47.740908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=20 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:28.167268 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15176cca-81f3-494c-933d-61f4de9457e7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:28.167788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15176cca-81f3-494c-933d-61f4de9457e7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:28.167788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15176cca-81f3-494c-933d-61f4de9457e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:28.187582 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-15176cca-81f3-494c-933d-61f4de9457e7 None None] MFA Rules not processed for user `43dd72b161f8442285dea411741d66b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:29:28.188783 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-15176cca-81f3-494c-933d-61f4de9457e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:28.241516 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1145/2299] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:29:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:29:31.319838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce35609-067b-4a9e-b249-94faaf6ca293 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:31.321290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0ce35609-067b-4a9e-b249-94faaf6ca293 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:31.391014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce35609-067b-4a9e-b249-94faaf6ca293 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:31.395924 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199cc0f-38a4-4ebd-ab61-84ce125ccbfe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f62745b92a864480a9438de581e1d05e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:31.396518 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199cc0f-38a4-4ebd-ab61-84ce125ccbfe service placement] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:31.396794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199cc0f-38a4-4ebd-ab61-84ce125ccbfe service placement] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:31.397064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199cc0f-38a4-4ebd-ab61-84ce125ccbfe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:31.398774 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3199cc0f-38a4-4ebd-ab61-84ce125ccbfe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:31.410336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199cc0f-38a4-4ebd-ab61-84ce125ccbfe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:31.412116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199cc0f-38a4-4ebd-ab61-84ce125ccbfe service placement] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:31.423722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1155/2300] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:29:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:29:33.176448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15176cca-81f3-494c-933d-61f4de9457e7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:33.178599 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-15176cca-81f3-494c-933d-61f4de9457e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:33.265294 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15176cca-81f3-494c-933d-61f4de9457e7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:33.277138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6ec3a-81b5-4eb7-b70f-94022df82847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:33.277778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6ec3a-81b5-4eb7-b70f-94022df82847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:33.278053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6ec3a-81b5-4eb7-b70f-94022df82847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:33.278353 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6ec3a-81b5-4eb7-b70f-94022df82847 admin admin] PATH_INFO: `/v3/users/97ad561b75214de8bfdf7cde1508d752` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:33.279489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6ec3a-81b5-4eb7-b70f-94022df82847 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97ad561b75214de8bfdf7cde1508d752)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:33.282062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6ec3a-81b5-4eb7-b70f-94022df82847 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:33.352654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-04e6ec3a-81b5-4eb7-b70f-94022df82847 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97ad561b75214de8bfdf7cde1508d752'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:33.363627 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-04e6ec3a-81b5-4eb7-b70f-94022df82847 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97ad561b75214de8bfdf7cde1508d752'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:33.376945 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-04e6ec3a-81b5-4eb7-b70f-94022df82847 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97ad561b75214de8bfdf7cde1508d752'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:33.377819 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-04e6ec3a-81b5-4eb7-b70f-94022df82847 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97ad561b75214de8bfdf7cde1508d752'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:33.378853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-04e6ec3a-81b5-4eb7-b70f-94022df82847 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97ad561b75214de8bfdf7cde1508d752'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:33.403758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1146/2301] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:29:33 2022] DELETE /identity/v3/users/97ad561b75214de8bfdf7cde1508d752 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:29:33.408842 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60002f77-59e1-4bfd-8bfe-c1160b9fc250 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:33.409337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60002f77-59e1-4bfd-8bfe-c1160b9fc250 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:33.409591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60002f77-59e1-4bfd-8bfe-c1160b9fc250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:33.430182 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-60002f77-59e1-4bfd-8bfe-c1160b9fc250 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:29:33.431576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-60002f77-59e1-4bfd-8bfe-c1160b9fc250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:33.504790 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1156/2302] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:29:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:29:33.514554 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6ec3a-81b5-4eb7-b70f-94022df82847 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:33.516729 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-04e6ec3a-81b5-4eb7-b70f-94022df82847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:33.546421 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:33.587436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6ec3a-81b5-4eb7-b70f-94022df82847 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:33.591813 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c01cd3-4e9f-4e44-9271-61fe063bd4dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:33.592509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c01cd3-4e9f-4e44-9271-61fe063bd4dd service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:33.592854 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c01cd3-4e9f-4e44-9271-61fe063bd4dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:33.593249 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c01cd3-4e9f-4e44-9271-61fe063bd4dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:33.604739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c01cd3-4e9f-4e44-9271-61fe063bd4dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:33.606359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c01cd3-4e9f-4e44-9271-61fe063bd4dd service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:33.622479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1147/2303] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:29:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:29:33.915891 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60002f77-59e1-4bfd-8bfe-c1160b9fc250 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:33.917572 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-60002f77-59e1-4bfd-8bfe-c1160b9fc250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:33.941341 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60002f77-59e1-4bfd-8bfe-c1160b9fc250 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:33.948598 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a64f10-2856-45b1-a535-536a327f48ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:33.949322 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a64f10-2856-45b1-a535-536a327f48ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:33.949582 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a64f10-2856-45b1-a535-536a327f48ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:33.949953 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a64f10-2856-45b1-a535-536a327f48ae admin admin] PATH_INFO: `/v3/projects/e7edd39524494605ae420bee713126e4` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:33.951005 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a64f10-2856-45b1-a535-536a327f48ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7edd39524494605ae420bee713126e4)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:33.953144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a64f10-2856-45b1-a535-536a327f48ae admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:34.002609 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e9a64f10-2856-45b1-a535-536a327f48ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7edd39524494605ae420bee713126e4', 'request_id': 'req-e9a64f10-2856-45b1-a535-536a327f48ae'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:34.008258 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e9a64f10-2856-45b1-a535-536a327f48ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7edd39524494605ae420bee713126e4', 'request_id': 'req-e9a64f10-2856-45b1-a535-536a327f48ae'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:34.014969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e9a64f10-2856-45b1-a535-536a327f48ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7edd39524494605ae420bee713126e4', 'request_id': 'req-e9a64f10-2856-45b1-a535-536a327f48ae'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:34.026134 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e9a64f10-2856-45b1-a535-536a327f48ae admin admin] The token cache is being invalidate because project e7edd39524494605ae420bee713126e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:29:34.026455 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e9a64f10-2856-45b1-a535-536a327f48ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:34.027102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e9a64f10-2856-45b1-a535-536a327f48ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:34.028683 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1157/2304] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:29:33 2022] DELETE /identity/v3/projects/e7edd39524494605ae420bee713126e4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:29:34.033725 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f500ac7-f927-42df-9bc9-edf98352f50b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:34.034014 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f500ac7-f927-42df-9bc9-edf98352f50b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:34.034319 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f500ac7-f927-42df-9bc9-edf98352f50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:34.054042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2f500ac7-f927-42df-9bc9-edf98352f50b None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:29:34.055360 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2f500ac7-f927-42df-9bc9-edf98352f50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:34.117156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1148/2305] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:29:34.126445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75726489-5fbb-4e27-b9ea-07d163205961 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:34.126762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75726489-5fbb-4e27-b9ea-07d163205961 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:34.127039 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75726489-5fbb-4e27-b9ea-07d163205961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:34.146645 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-75726489-5fbb-4e27-b9ea-07d163205961 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:29:34.147870 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-75726489-5fbb-4e27-b9ea-07d163205961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:34.155979 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1158/2306] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:29:34.160091 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f500ac7-f927-42df-9bc9-edf98352f50b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:34.171436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f500ac7-f927-42df-9bc9-edf98352f50b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:34.176054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286c3e7-3df4-40e7-a05b-734b7c238dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:34.176655 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286c3e7-3df4-40e7-a05b-734b7c238dcc admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:34.177260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286c3e7-3df4-40e7-a05b-734b7c238dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:34.177260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286c3e7-3df4-40e7-a05b-734b7c238dcc admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:34.178039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286c3e7-3df4-40e7-a05b-734b7c238dcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:29:34.178537 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286c3e7-3df4-40e7-a05b-734b7c238dcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:34.180333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286c3e7-3df4-40e7-a05b-734b7c238dcc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:34.193792 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1149/2307] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:29:34.198027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75726489-5fbb-4e27-b9ea-07d163205961 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:34.207690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75726489-5fbb-4e27-b9ea-07d163205961 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:34.212431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fe17d-768e-4d2a-94dc-9a207ca5bc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:34.213022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fe17d-768e-4d2a-94dc-9a207ca5bc24 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:34.213300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fe17d-768e-4d2a-94dc-9a207ca5bc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:34.213569 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fe17d-768e-4d2a-94dc-9a207ca5bc24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:34.214190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fe17d-768e-4d2a-94dc-9a207ca5bc24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:34.216170 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fe17d-768e-4d2a-94dc-9a207ca5bc24 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:34.220749 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:34.250126 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1159/2308] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:29:34 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:29:34.254501 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286c3e7-3df4-40e7-a05b-734b7c238dcc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:34.265377 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286c3e7-3df4-40e7-a05b-734b7c238dcc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:34.324001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59227ae-6d17-4027-accc-cfaca8b91636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:34.324631 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59227ae-6d17-4027-accc-cfaca8b91636 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:34.324937 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59227ae-6d17-4027-accc-cfaca8b91636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:34.325203 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59227ae-6d17-4027-accc-cfaca8b91636 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:34.325819 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59227ae-6d17-4027-accc-cfaca8b91636 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:34.327829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59227ae-6d17-4027-accc-cfaca8b91636 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:34.352931 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1150/2309] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:29:34 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:29:34.354367 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fe17d-768e-4d2a-94dc-9a207ca5bc24 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:34.365610 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fe17d-768e-4d2a-94dc-9a207ca5bc24 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:34.371898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d79616-c478-4981-b163-36adfcfc8bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:34.372491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d79616-c478-4981-b163-36adfcfc8bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:34.372764 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d79616-c478-4981-b163-36adfcfc8bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:34.373221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d79616-c478-4981-b163-36adfcfc8bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:34.373764 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d79616-c478-4981-b163-36adfcfc8bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:34.375467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d79616-c478-4981-b163-36adfcfc8bc5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:34.387791 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1160/2310] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:29:34.391904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59227ae-6d17-4027-accc-cfaca8b91636 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:34.402476 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59227ae-6d17-4027-accc-cfaca8b91636 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:34.408015 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540237b-b819-49ca-a7df-144df356cdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:34.408768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540237b-b819-49ca-a7df-144df356cdd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:34.409075 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540237b-b819-49ca-a7df-144df356cdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:34.409343 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540237b-b819-49ca-a7df-144df356cdd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:34.409978 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540237b-b819-49ca-a7df-144df356cdd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:34.411624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540237b-b819-49ca-a7df-144df356cdd5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:34.430506 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1151/2311] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:29:34.434400 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d79616-c478-4981-b163-36adfcfc8bc5 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:34.448969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d79616-c478-4981-b163-36adfcfc8bc5 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:34.453905 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0265d88-45e7-4254-bd25-990cef9ea6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:34.454594 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0265d88-45e7-4254-bd25-990cef9ea6aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:34.454880 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0265d88-45e7-4254-bd25-990cef9ea6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:34.455157 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0265d88-45e7-4254-bd25-990cef9ea6aa admin admin] PATH_INFO: `/v3/projects/47f30291a2dc4f8284c2e14f8b0818c7/users/160122487dc04689beb58852c81a9fe4/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:34.474183 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0265d88-45e7-4254-bd25-990cef9ea6aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=47f30291a2dc4f8284c2e14f8b0818c7, user_id=160122487dc04689beb58852c81a9fe4, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:34.476960 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0265d88-45e7-4254-bd25-990cef9ea6aa admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:34.491068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1161/2312] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:29:34 2022] PUT /identity/v3/projects/47f30291a2dc4f8284c2e14f8b0818c7/users/160122487dc04689beb58852c81a9fe4/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:29:34.501412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8d819-64bc-4901-ad69-d496912e7117 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:34.501760 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8d819-64bc-4901-ad69-d496912e7117 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:34.502090 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8d819-64bc-4901-ad69-d496912e7117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:34.529387 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-09d8d819-64bc-4901-ad69-d496912e7117 None None] MFA Rules not processed for user `160122487dc04689beb58852c81a9fe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:29:34.531139 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-09d8d819-64bc-4901-ad69-d496912e7117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:34.551127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:34.596574 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1152/2313] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:29:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:29:34.607743 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0265d88-45e7-4254-bd25-990cef9ea6aa admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:34.610023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a0265d88-45e7-4254-bd25-990cef9ea6aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:34.666363 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0265d88-45e7-4254-bd25-990cef9ea6aa admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:34.669987 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c8a21-7caa-499c-9fcc-eea919c43dd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:34.670607 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c8a21-7caa-499c-9fcc-eea919c43dd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:34.670889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c8a21-7caa-499c-9fcc-eea919c43dd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:34.671165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c8a21-7caa-499c-9fcc-eea919c43dd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:34.679619 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c8a21-7caa-499c-9fcc-eea919c43dd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:34.681064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c8a21-7caa-499c-9fcc-eea919c43dd9 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:34.690140 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1162/2314] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:29:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:29:41.548616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d8d819-64bc-4901-ad69-d496912e7117 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:41.614863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d8d819-64bc-4901-ad69-d496912e7117 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:41.619663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b408fed-b50f-486b-83ff-e1cbbdc59a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:41.620423 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b408fed-b50f-486b-83ff-e1cbbdc59a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:41.620832 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b408fed-b50f-486b-83ff-e1cbbdc59a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:41.621108 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b408fed-b50f-486b-83ff-e1cbbdc59a5f admin admin] PATH_INFO: `/v3/users/160122487dc04689beb58852c81a9fe4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:41.622319 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b408fed-b50f-486b-83ff-e1cbbdc59a5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=160122487dc04689beb58852c81a9fe4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:41.627180 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b408fed-b50f-486b-83ff-e1cbbdc59a5f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:41.689028 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0b408fed-b50f-486b-83ff-e1cbbdc59a5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '160122487dc04689beb58852c81a9fe4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:41.702813 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0b408fed-b50f-486b-83ff-e1cbbdc59a5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '160122487dc04689beb58852c81a9fe4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:41.710941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0b408fed-b50f-486b-83ff-e1cbbdc59a5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '160122487dc04689beb58852c81a9fe4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:41.711779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0b408fed-b50f-486b-83ff-e1cbbdc59a5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '160122487dc04689beb58852c81a9fe4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:41.712461 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0b408fed-b50f-486b-83ff-e1cbbdc59a5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '160122487dc04689beb58852c81a9fe4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:41.738312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1153/2315] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:29:41 2022] DELETE /identity/v3/users/160122487dc04689beb58852c81a9fe4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:29:41.743060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca550d9-2e1a-48ec-9151-6eb5276e2b57 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:41.743514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca550d9-2e1a-48ec-9151-6eb5276e2b57 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:41.743813 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca550d9-2e1a-48ec-9151-6eb5276e2b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:41.770029 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6ca550d9-2e1a-48ec-9151-6eb5276e2b57 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:29:41.771393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6ca550d9-2e1a-48ec-9151-6eb5276e2b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:41.847854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1163/2316] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:29:41.857454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b408fed-b50f-486b-83ff-e1cbbdc59a5f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:41.859019 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0b408fed-b50f-486b-83ff-e1cbbdc59a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:41.911779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:41.927133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b408fed-b50f-486b-83ff-e1cbbdc59a5f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:41.931095 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b16418-a512-487d-949f-77ab5746b83f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:41.931701 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b16418-a512-487d-949f-77ab5746b83f service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:41.931981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b16418-a512-487d-949f-77ab5746b83f service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:41.932253 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b16418-a512-487d-949f-77ab5746b83f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:41.941504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b16418-a512-487d-949f-77ab5746b83f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:41.942984 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b16418-a512-487d-949f-77ab5746b83f service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:41.952380 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1154/2317] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:29:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:29:42.226940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca550d9-2e1a-48ec-9151-6eb5276e2b57 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:42.228589 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6ca550d9-2e1a-48ec-9151-6eb5276e2b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:42.241586 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca550d9-2e1a-48ec-9151-6eb5276e2b57 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:42.247742 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b7127-eb81-4b78-a2e9-f865d1be3f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:42.248370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b7127-eb81-4b78-a2e9-f865d1be3f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:42.248729 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b7127-eb81-4b78-a2e9-f865d1be3f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:42.248904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b7127-eb81-4b78-a2e9-f865d1be3f76 admin admin] PATH_INFO: `/v3/projects/47f30291a2dc4f8284c2e14f8b0818c7` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:42.250093 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b7127-eb81-4b78-a2e9-f865d1be3f76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47f30291a2dc4f8284c2e14f8b0818c7)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:42.253193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b7127-eb81-4b78-a2e9-f865d1be3f76 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:42.304204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:42.305584 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-017b7127-eb81-4b78-a2e9-f865d1be3f76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47f30291a2dc4f8284c2e14f8b0818c7', 'request_id': 'req-017b7127-eb81-4b78-a2e9-f865d1be3f76'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:42.312890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-017b7127-eb81-4b78-a2e9-f865d1be3f76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47f30291a2dc4f8284c2e14f8b0818c7', 'request_id': 'req-017b7127-eb81-4b78-a2e9-f865d1be3f76'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:42.321046 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-017b7127-eb81-4b78-a2e9-f865d1be3f76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47f30291a2dc4f8284c2e14f8b0818c7', 'request_id': 'req-017b7127-eb81-4b78-a2e9-f865d1be3f76'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:42.333437 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-017b7127-eb81-4b78-a2e9-f865d1be3f76 admin admin] The token cache is being invalidate because project 47f30291a2dc4f8284c2e14f8b0818c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:29:42.333727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-017b7127-eb81-4b78-a2e9-f865d1be3f76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:42.334472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-017b7127-eb81-4b78-a2e9-f865d1be3f76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:42.335983 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1164/2318] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:29:42 2022] DELETE /identity/v3/projects/47f30291a2dc4f8284c2e14f8b0818c7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:29:42.340900 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d491b-d2ae-4590-ad95-ab5078935143 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:42.341205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d491b-d2ae-4590-ad95-ab5078935143 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:42.341473 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d491b-d2ae-4590-ad95-ab5078935143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:42.361423 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-212d491b-d2ae-4590-ad95-ab5078935143 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:29:42.362665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-212d491b-d2ae-4590-ad95-ab5078935143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:42.422492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1155/2319] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:29:42.431203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065dbcb-3bdd-4e2f-8374-f220bfd1a080 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:42.431495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065dbcb-3bdd-4e2f-8374-f220bfd1a080 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:42.431777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065dbcb-3bdd-4e2f-8374-f220bfd1a080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:42.453501 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-3065dbcb-3bdd-4e2f-8374-f220bfd1a080 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:29:42.454760 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3065dbcb-3bdd-4e2f-8374-f220bfd1a080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:42.462923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1165/2320] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:29:42.467154 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d491b-d2ae-4590-ad95-ab5078935143 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:42.477349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d491b-d2ae-4590-ad95-ab5078935143 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:42.481867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cb814-26be-48e9-ad28-4f27bd69e9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:42.482479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cb814-26be-48e9-ad28-4f27bd69e9ff admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:42.482759 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cb814-26be-48e9-ad28-4f27bd69e9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:42.483023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cb814-26be-48e9-ad28-4f27bd69e9ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:42.483656 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74cb814-26be-48e9-ad28-4f27bd69e9ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:29:42.483954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74cb814-26be-48e9-ad28-4f27bd69e9ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:42.485576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74cb814-26be-48e9-ad28-4f27bd69e9ff admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:42.496908 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1156/2321] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:29:42.501611 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065dbcb-3bdd-4e2f-8374-f220bfd1a080 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:42.510782 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065dbcb-3bdd-4e2f-8374-f220bfd1a080 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:42.516076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ce874-001f-4cd3-9f26-470810bab901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:42.516727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ce874-001f-4cd3-9f26-470810bab901 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:42.517011 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ce874-001f-4cd3-9f26-470810bab901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:42.517333 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ce874-001f-4cd3-9f26-470810bab901 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:42.517971 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ce874-001f-4cd3-9f26-470810bab901 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:42.519970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ce874-001f-4cd3-9f26-470810bab901 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:42.550769 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1166/2322] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:29:42 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:29:42.552178 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cb814-26be-48e9-ad28-4f27bd69e9ff admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:42.562898 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cb814-26be-48e9-ad28-4f27bd69e9ff admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:42.623667 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1c139-9873-4637-82ad-493cdf6b8e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:42.624179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1c139-9873-4637-82ad-493cdf6b8e03 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:42.624454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1c139-9873-4637-82ad-493cdf6b8e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:42.624720 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1c139-9873-4637-82ad-493cdf6b8e03 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:42.625845 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1c139-9873-4637-82ad-493cdf6b8e03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:42.627791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1c139-9873-4637-82ad-493cdf6b8e03 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:42.651027 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1157/2323] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:29:42 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:29:42.655122 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ce874-001f-4cd3-9f26-470810bab901 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:42.666486 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ce874-001f-4cd3-9f26-470810bab901 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:42.671363 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132c839-7260-45ee-a080-418a62a3f594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:42.672005 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132c839-7260-45ee-a080-418a62a3f594 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:42.672286 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132c839-7260-45ee-a080-418a62a3f594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:42.672560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132c839-7260-45ee-a080-418a62a3f594 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:42.673206 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132c839-7260-45ee-a080-418a62a3f594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:42.674836 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132c839-7260-45ee-a080-418a62a3f594 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:42.685313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1167/2324] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:29:42.690038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1c139-9873-4637-82ad-493cdf6b8e03 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:42.700301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1c139-9873-4637-82ad-493cdf6b8e03 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:42.705940 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5ee16-e3b8-4571-a052-5a95fda9b605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:42.706577 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5ee16-e3b8-4571-a052-5a95fda9b605 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:42.706860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5ee16-e3b8-4571-a052-5a95fda9b605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:42.707134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5ee16-e3b8-4571-a052-5a95fda9b605 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:42.707779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5ee16-e3b8-4571-a052-5a95fda9b605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:42.709395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5ee16-e3b8-4571-a052-5a95fda9b605 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:42.720489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1158/2325] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:29:42.724720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132c839-7260-45ee-a080-418a62a3f594 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:42.734840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132c839-7260-45ee-a080-418a62a3f594 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:42.739528 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c584ff-48cb-4397-bb61-4540d343b6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:42.740181 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c584ff-48cb-4397-bb61-4540d343b6a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:42.740454 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c584ff-48cb-4397-bb61-4540d343b6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:42.740721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c584ff-48cb-4397-bb61-4540d343b6a0 admin admin] PATH_INFO: `/v3/projects/4bbe370b33ee469293b93fcd8b99d72f/users/c9f7f471f9394c4eb411e2089be4a69e/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:42.756063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c584ff-48cb-4397-bb61-4540d343b6a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bbe370b33ee469293b93fcd8b99d72f, user_id=c9f7f471f9394c4eb411e2089be4a69e, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:42.758770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c584ff-48cb-4397-bb61-4540d343b6a0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:42.781350 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1168/2326] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:29:42 2022] PUT /identity/v3/projects/4bbe370b33ee469293b93fcd8b99d72f/users/c9f7f471f9394c4eb411e2089be4a69e/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:29:42.791345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a760f1c-965d-4031-a4dd-3a0ba240ed0f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:42.791645 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a760f1c-965d-4031-a4dd-3a0ba240ed0f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:42.791894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a760f1c-965d-4031-a4dd-3a0ba240ed0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:42.821823 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5a760f1c-965d-4031-a4dd-3a0ba240ed0f None None] MFA Rules not processed for user `c9f7f471f9394c4eb411e2089be4a69e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:29:42.823178 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5a760f1c-965d-4031-a4dd-3a0ba240ed0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:42.879170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:42.923499 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1159/2327] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:29:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:29:42.930925 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c584ff-48cb-4397-bb61-4540d343b6a0 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:42.932668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f2c584ff-48cb-4397-bb61-4540d343b6a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:43.009640 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c584ff-48cb-4397-bb61-4540d343b6a0 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:43.015588 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df21cf8-2dbf-4418-bc49-03863d199c96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:43.016421 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df21cf8-2dbf-4418-bc49-03863d199c96 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:43.016811 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df21cf8-2dbf-4418-bc49-03863d199c96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:43.017157 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df21cf8-2dbf-4418-bc49-03863d199c96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:43.026580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df21cf8-2dbf-4418-bc49-03863d199c96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:43.028094 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df21cf8-2dbf-4418-bc49-03863d199c96 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:43.039617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1169/2328] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:29:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:29:52.911980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=9 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:53.035237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=60,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:18.055479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a760f1c-965d-4031-a4dd-3a0ba240ed0f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:18.119268 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a760f1c-965d-4031-a4dd-3a0ba240ed0f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:18.123735 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67dace-8904-4eca-bc3d-264cb0bfd83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:18.124350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67dace-8904-4eca-bc3d-264cb0bfd83d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:18.124628 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67dace-8904-4eca-bc3d-264cb0bfd83d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:18.124900 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67dace-8904-4eca-bc3d-264cb0bfd83d admin admin] PATH_INFO: `/v3/users/c9f7f471f9394c4eb411e2089be4a69e` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:18.126009 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67dace-8904-4eca-bc3d-264cb0bfd83d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9f7f471f9394c4eb411e2089be4a69e)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:18.127980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67dace-8904-4eca-bc3d-264cb0bfd83d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:18.173961 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5b67dace-8904-4eca-bc3d-264cb0bfd83d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9f7f471f9394c4eb411e2089be4a69e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:18.180924 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5b67dace-8904-4eca-bc3d-264cb0bfd83d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9f7f471f9394c4eb411e2089be4a69e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:18.188355 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5b67dace-8904-4eca-bc3d-264cb0bfd83d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9f7f471f9394c4eb411e2089be4a69e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:18.188989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5b67dace-8904-4eca-bc3d-264cb0bfd83d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9f7f471f9394c4eb411e2089be4a69e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:18.189867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5b67dace-8904-4eca-bc3d-264cb0bfd83d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9f7f471f9394c4eb411e2089be4a69e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:18.207988 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1160/2329] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:30:18 2022] DELETE /identity/v3/users/c9f7f471f9394c4eb411e2089be4a69e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:30:18.212580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db891aa-273a-496b-a564-ca67809e3324 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:18.213055 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db891aa-273a-496b-a564-ca67809e3324 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:18.213179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db891aa-273a-496b-a564-ca67809e3324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:18.232239 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-1db891aa-273a-496b-a564-ca67809e3324 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:18.233442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1db891aa-273a-496b-a564-ca67809e3324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:18.291419 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1170/2330] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:18.303985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67dace-8904-4eca-bc3d-264cb0bfd83d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:18.305534 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5b67dace-8904-4eca-bc3d-264cb0bfd83d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:18.362891 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67dace-8904-4eca-bc3d-264cb0bfd83d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:18.366584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42034c23-ef04-411b-8625-d8478e7dbd4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:18.367182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42034c23-ef04-411b-8625-d8478e7dbd4d service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:18.367460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42034c23-ef04-411b-8625-d8478e7dbd4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:18.367730 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42034c23-ef04-411b-8625-d8478e7dbd4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:18.376734 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42034c23-ef04-411b-8625-d8478e7dbd4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:18.378166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42034c23-ef04-411b-8625-d8478e7dbd4d service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:18.387702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1161/2331] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:18.675268 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db891aa-273a-496b-a564-ca67809e3324 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:18.676873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1db891aa-273a-496b-a564-ca67809e3324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:18.691054 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db891aa-273a-496b-a564-ca67809e3324 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:18.696091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdaf0d6-ee97-43bd-9a4e-8bd9ea45b50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:18.696842 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaf0d6-ee97-43bd-9a4e-8bd9ea45b50f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:18.697128 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaf0d6-ee97-43bd-9a4e-8bd9ea45b50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:18.697404 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaf0d6-ee97-43bd-9a4e-8bd9ea45b50f admin admin] PATH_INFO: `/v3/projects/4bbe370b33ee469293b93fcd8b99d72f` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:18.698596 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdaf0d6-ee97-43bd-9a4e-8bd9ea45b50f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bbe370b33ee469293b93fcd8b99d72f)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:18.700467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdaf0d6-ee97-43bd-9a4e-8bd9ea45b50f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:18.752021 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2cdaf0d6-ee97-43bd-9a4e-8bd9ea45b50f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bbe370b33ee469293b93fcd8b99d72f', 'request_id': 'req-2cdaf0d6-ee97-43bd-9a4e-8bd9ea45b50f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:18.757654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2cdaf0d6-ee97-43bd-9a4e-8bd9ea45b50f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bbe370b33ee469293b93fcd8b99d72f', 'request_id': 'req-2cdaf0d6-ee97-43bd-9a4e-8bd9ea45b50f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:18.765956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2cdaf0d6-ee97-43bd-9a4e-8bd9ea45b50f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bbe370b33ee469293b93fcd8b99d72f', 'request_id': 'req-2cdaf0d6-ee97-43bd-9a4e-8bd9ea45b50f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:18.777455 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2cdaf0d6-ee97-43bd-9a4e-8bd9ea45b50f admin admin] The token cache is being invalidate because project 4bbe370b33ee469293b93fcd8b99d72f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:30:18.777746 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2cdaf0d6-ee97-43bd-9a4e-8bd9ea45b50f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:18.778423 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2cdaf0d6-ee97-43bd-9a4e-8bd9ea45b50f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:18.779941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1171/2332] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:30:18 2022] DELETE /identity/v3/projects/4bbe370b33ee469293b93fcd8b99d72f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:30:18.784806 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7120cf9-0def-49c0-a75d-6e8fc710a93f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:18.785091 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7120cf9-0def-49c0-a75d-6e8fc710a93f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:18.785359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7120cf9-0def-49c0-a75d-6e8fc710a93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:18.804654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a7120cf9-0def-49c0-a75d-6e8fc710a93f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:18.805776 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a7120cf9-0def-49c0-a75d-6e8fc710a93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:18.823785 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:18.868773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1162/2333] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:18.878508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f6db7-0304-4173-a164-1f9510f148c8 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:18.878858 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f6db7-0304-4173-a164-1f9510f148c8 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:18.879199 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f6db7-0304-4173-a164-1f9510f148c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:18.898661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-942f6db7-0304-4173-a164-1f9510f148c8 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:18.899872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-942f6db7-0304-4173-a164-1f9510f148c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:18.907843 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1172/2334] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:18.912115 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7120cf9-0def-49c0-a75d-6e8fc710a93f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:18.922380 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7120cf9-0def-49c0-a75d-6e8fc710a93f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:18.927086 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84812d45-941a-4474-8984-25097c0809e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:18.927677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84812d45-941a-4474-8984-25097c0809e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:18.927952 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84812d45-941a-4474-8984-25097c0809e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:18.928217 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84812d45-941a-4474-8984-25097c0809e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:18.928856 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84812d45-941a-4474-8984-25097c0809e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:30:18.929156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84812d45-941a-4474-8984-25097c0809e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:18.930790 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84812d45-941a-4474-8984-25097c0809e2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:18.946337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1163/2335] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:30:18.947948 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f6db7-0304-4173-a164-1f9510f148c8 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:18.957425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f6db7-0304-4173-a164-1f9510f148c8 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:18.962171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6cf68-6446-48be-8702-786a760ab201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:18.962887 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6cf68-6446-48be-8702-786a760ab201 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:18.963233 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6cf68-6446-48be-8702-786a760ab201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:18.963571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6cf68-6446-48be-8702-786a760ab201 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:18.964272 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6cf68-6446-48be-8702-786a760ab201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:18.966210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6cf68-6446-48be-8702-786a760ab201 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:18.995068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1173/2336] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:30:18 2022] POST /identity/v3/projects => generated 399 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:30:18.998999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84812d45-941a-4474-8984-25097c0809e2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:19.010901 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84812d45-941a-4474-8984-25097c0809e2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:19.082289 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43675305-c96a-44e7-b445-6cfd9d65ea30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:19.083115 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43675305-c96a-44e7-b445-6cfd9d65ea30 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:19.083390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43675305-c96a-44e7-b445-6cfd9d65ea30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:19.083689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43675305-c96a-44e7-b445-6cfd9d65ea30 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:19.084307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43675305-c96a-44e7-b445-6cfd9d65ea30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:19.086212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43675305-c96a-44e7-b445-6cfd9d65ea30 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:19.112819 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1164/2337] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:30:18 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:30:19.114037 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6cf68-6446-48be-8702-786a760ab201 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:19.125513 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6cf68-6446-48be-8702-786a760ab201 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:19.130485 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44499bd7-539b-4f47-be51-fd0d1886d2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:19.131095 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44499bd7-539b-4f47-be51-fd0d1886d2cb admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:19.131372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44499bd7-539b-4f47-be51-fd0d1886d2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:19.131648 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44499bd7-539b-4f47-be51-fd0d1886d2cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:19.132405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44499bd7-539b-4f47-be51-fd0d1886d2cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:19.134064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44499bd7-539b-4f47-be51-fd0d1886d2cb admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:19.145371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1174/2338] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:30:19.149412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43675305-c96a-44e7-b445-6cfd9d65ea30 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:19.161118 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43675305-c96a-44e7-b445-6cfd9d65ea30 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:19.166010 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7212d-21e3-41a2-8412-8064fef2dbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:19.166642 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7212d-21e3-41a2-8412-8064fef2dbf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:19.166921 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7212d-21e3-41a2-8412-8064fef2dbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:19.167193 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7212d-21e3-41a2-8412-8064fef2dbf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:19.167829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7212d-21e3-41a2-8412-8064fef2dbf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:19.169428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7212d-21e3-41a2-8412-8064fef2dbf4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:19.180751 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1165/2339] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:30:19.185068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44499bd7-539b-4f47-be51-fd0d1886d2cb admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:19.196102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44499bd7-539b-4f47-be51-fd0d1886d2cb admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:19.200838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4939a-d9d6-40c3-9ff8-63b957e66214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:19.201470 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4939a-d9d6-40c3-9ff8-63b957e66214 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:19.201751 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4939a-d9d6-40c3-9ff8-63b957e66214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:19.202024 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4939a-d9d6-40c3-9ff8-63b957e66214 admin admin] PATH_INFO: `/v3/projects/c1afb6d04ae94d06b6f82405907a62dd/users/bcb365c368b34f3d8444fca7abf5be4e/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:19.217073 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4939a-d9d6-40c3-9ff8-63b957e66214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1afb6d04ae94d06b6f82405907a62dd, user_id=bcb365c368b34f3d8444fca7abf5be4e, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:19.220333 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4939a-d9d6-40c3-9ff8-63b957e66214 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:19.233460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1175/2340] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:30:19 2022] PUT /identity/v3/projects/c1afb6d04ae94d06b6f82405907a62dd/users/bcb365c368b34f3d8444fca7abf5be4e/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:30:19.244128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46efd4-8c42-4868-8191-01e0f83e2ede None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:19.244412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46efd4-8c42-4868-8191-01e0f83e2ede None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:19.244683 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46efd4-8c42-4868-8191-01e0f83e2ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:19.273000 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0f46efd4-8c42-4868-8191-01e0f83e2ede None None] MFA Rules not processed for user `bcb365c368b34f3d8444fca7abf5be4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:19.274217 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0f46efd4-8c42-4868-8191-01e0f83e2ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:19.339857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1166/2341] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:30:19 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:19.351495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4939a-d9d6-40c3-9ff8-63b957e66214 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:19.353188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-05f4939a-d9d6-40c3-9ff8-63b957e66214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:19.378568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:19.416568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4939a-d9d6-40c3-9ff8-63b957e66214 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:19.423341 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f65c2a-9091-4b9d-bcaa-fa78909fbeae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:19.423936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f65c2a-9091-4b9d-bcaa-fa78909fbeae service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:19.424218 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f65c2a-9091-4b9d-bcaa-fa78909fbeae service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:19.424497 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f65c2a-9091-4b9d-bcaa-fa78909fbeae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:19.433338 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f65c2a-9091-4b9d-bcaa-fa78909fbeae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:19.434820 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f65c2a-9091-4b9d-bcaa-fa78909fbeae service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:19.444852 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1176/2342] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:19 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:22.545395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45dd73-2c37-4dec-8fb6-d9450780b320 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:22.545882 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45dd73-2c37-4dec-8fb6-d9450780b320 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:22.545977 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45dd73-2c37-4dec-8fb6-d9450780b320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:22.565732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9e45dd73-2c37-4dec-8fb6-d9450780b320 None None] MFA Rules not processed for user `ef2ca1a30a76468cbe02b20efd6ca37e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:22.566969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9e45dd73-2c37-4dec-8fb6-d9450780b320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:22.579052 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:22.628340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1167/2343] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:30:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:26.516107 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f65c2a-9091-4b9d-bcaa-fa78909fbeae service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:26.517559 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-43f65c2a-9091-4b9d-bcaa-fa78909fbeae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:26.573267 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f65c2a-9091-4b9d-bcaa-fa78909fbeae service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:26.576835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b42b13-375f-4ad1-a660-4946e816e70b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:26.577428 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b42b13-375f-4ad1-a660-4946e816e70b service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:26.577704 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b42b13-375f-4ad1-a660-4946e816e70b service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:26.577974 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b42b13-375f-4ad1-a660-4946e816e70b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:26.579679 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b0b42b13-375f-4ad1-a660-4946e816e70b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:26.648841 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b42b13-375f-4ad1-a660-4946e816e70b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:26.650271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b42b13-375f-4ad1-a660-4946e816e70b service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:26.661107 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1177/2344] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:27.771884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45dd73-2c37-4dec-8fb6-d9450780b320 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:27.773369 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9e45dd73-2c37-4dec-8fb6-d9450780b320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:27.785207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45dd73-2c37-4dec-8fb6-d9450780b320 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:27.788820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aefdea-bd9c-435d-b49d-21a3fa0ae802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:27.789412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aefdea-bd9c-435d-b49d-21a3fa0ae802 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:27.789686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aefdea-bd9c-435d-b49d-21a3fa0ae802 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:27.789953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aefdea-bd9c-435d-b49d-21a3fa0ae802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:27.791639 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-37aefdea-bd9c-435d-b49d-21a3fa0ae802 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:27.804516 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aefdea-bd9c-435d-b49d-21a3fa0ae802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:27.805943 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aefdea-bd9c-435d-b49d-21a3fa0ae802 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:27.817332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1168/2345] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:29.284349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b42b13-375f-4ad1-a660-4946e816e70b service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:29.285901 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b0b42b13-375f-4ad1-a660-4946e816e70b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:29.352719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b42b13-375f-4ad1-a660-4946e816e70b service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:29.357434 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63772df4-e74a-4123-a9ac-35fe996ce8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:29.358049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63772df4-e74a-4123-a9ac-35fe996ce8d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:29.358354 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63772df4-e74a-4123-a9ac-35fe996ce8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:29.358627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63772df4-e74a-4123-a9ac-35fe996ce8d5 admin admin] PATH_INFO: `/v3/users/eea82fd0321d4e88a7943681099aadd7` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:29.360275 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63772df4-e74a-4123-a9ac-35fe996ce8d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eea82fd0321d4e88a7943681099aadd7)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:29.362175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63772df4-e74a-4123-a9ac-35fe996ce8d5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:29.369574 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:29.428674 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-63772df4-e74a-4123-a9ac-35fe996ce8d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eea82fd0321d4e88a7943681099aadd7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:29.435513 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-63772df4-e74a-4123-a9ac-35fe996ce8d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eea82fd0321d4e88a7943681099aadd7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:29.448018 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-63772df4-e74a-4123-a9ac-35fe996ce8d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eea82fd0321d4e88a7943681099aadd7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:29.448960 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-63772df4-e74a-4123-a9ac-35fe996ce8d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eea82fd0321d4e88a7943681099aadd7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:29.450941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-63772df4-e74a-4123-a9ac-35fe996ce8d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eea82fd0321d4e88a7943681099aadd7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:29.479357 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1178/2346] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:30:29 2022] DELETE /identity/v3/users/eea82fd0321d4e88a7943681099aadd7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:30:29.480775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c49938-1926-46e6-aa66-00afee6304a3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:29.481232 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c49938-1926-46e6-aa66-00afee6304a3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:29.481325 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c49938-1926-46e6-aa66-00afee6304a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:29.501412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c7c49938-1926-46e6-aa66-00afee6304a3 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:29.503276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c7c49938-1926-46e6-aa66-00afee6304a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:29.577115 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1169/2347] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:30:29.585544 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63772df4-e74a-4123-a9ac-35fe996ce8d5 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:29.587814 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-63772df4-e74a-4123-a9ac-35fe996ce8d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:29.661960 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63772df4-e74a-4123-a9ac-35fe996ce8d5 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:29.665668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3c8c9-da76-4b31-b47c-61314e196df0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:29.666285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3c8c9-da76-4b31-b47c-61314e196df0 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:29.666568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3c8c9-da76-4b31-b47c-61314e196df0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:29.669021 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3c8c9-da76-4b31-b47c-61314e196df0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:29.676581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3c8c9-da76-4b31-b47c-61314e196df0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:29.677999 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3c8c9-da76-4b31-b47c-61314e196df0 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:29.690125 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1179/2348] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:30.128442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c49938-1926-46e6-aa66-00afee6304a3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:30.130732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c7c49938-1926-46e6-aa66-00afee6304a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:30.147351 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c49938-1926-46e6-aa66-00afee6304a3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:30.161136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80d5b0-d796-472d-87ac-12d51bfab573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:30.162052 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80d5b0-d796-472d-87ac-12d51bfab573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:30.162406 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80d5b0-d796-472d-87ac-12d51bfab573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:30.162745 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80d5b0-d796-472d-87ac-12d51bfab573 admin admin] PATH_INFO: `/v3/projects/427fbcf122944aca83622b7fd0616982` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:30.163904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80d5b0-d796-472d-87ac-12d51bfab573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=427fbcf122944aca83622b7fd0616982)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:30.165818 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80d5b0-d796-472d-87ac-12d51bfab573 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:30.216784 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ac80d5b0-d796-472d-87ac-12d51bfab573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '427fbcf122944aca83622b7fd0616982', 'request_id': 'req-ac80d5b0-d796-472d-87ac-12d51bfab573'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:30.223633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ac80d5b0-d796-472d-87ac-12d51bfab573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '427fbcf122944aca83622b7fd0616982', 'request_id': 'req-ac80d5b0-d796-472d-87ac-12d51bfab573'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:30.231937 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ac80d5b0-d796-472d-87ac-12d51bfab573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '427fbcf122944aca83622b7fd0616982', 'request_id': 'req-ac80d5b0-d796-472d-87ac-12d51bfab573'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:30.243520 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ac80d5b0-d796-472d-87ac-12d51bfab573 admin admin] The token cache is being invalidate because project 427fbcf122944aca83622b7fd0616982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:30:30.243810 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ac80d5b0-d796-472d-87ac-12d51bfab573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:30.244467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ac80d5b0-d796-472d-87ac-12d51bfab573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:30.245954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1170/2349] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:30:30 2022] DELETE /identity/v3/projects/427fbcf122944aca83622b7fd0616982 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:30:30.251081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888cb3a-7918-456f-b9ed-16eb7d5fbbef None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:30.251445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888cb3a-7918-456f-b9ed-16eb7d5fbbef None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:30.251719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888cb3a-7918-456f-b9ed-16eb7d5fbbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:30.270953 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-4888cb3a-7918-456f-b9ed-16eb7d5fbbef None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:30.272155 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4888cb3a-7918-456f-b9ed-16eb7d5fbbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:30.334328 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1180/2350] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:30.342046 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c0f67-e722-43d9-952e-4f57765dd583 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:30.342371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c0f67-e722-43d9-952e-4f57765dd583 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:30.342645 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c0f67-e722-43d9-952e-4f57765dd583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:30.362126 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6f7c0f67-e722-43d9-952e-4f57765dd583 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:30.363340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6f7c0f67-e722-43d9-952e-4f57765dd583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:30.371113 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1171/2351] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:30.375762 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888cb3a-7918-456f-b9ed-16eb7d5fbbef None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:30.385254 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888cb3a-7918-456f-b9ed-16eb7d5fbbef None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:30.389772 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c47ab-9562-4f99-b5cd-e73bdb62aad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:30.390386 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c47ab-9562-4f99-b5cd-e73bdb62aad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:30.390694 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c47ab-9562-4f99-b5cd-e73bdb62aad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:30.391027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c47ab-9562-4f99-b5cd-e73bdb62aad9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:30.391672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c47ab-9562-4f99-b5cd-e73bdb62aad9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:30:30.391978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c47ab-9562-4f99-b5cd-e73bdb62aad9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:30.393557 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c47ab-9562-4f99-b5cd-e73bdb62aad9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:30.405019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1181/2352] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:30:30.409219 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c0f67-e722-43d9-952e-4f57765dd583 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:30.418526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c0f67-e722-43d9-952e-4f57765dd583 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:30.423732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4291a-dc96-4965-80bc-5a7c4e5256d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:30.424322 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4291a-dc96-4965-80bc-5a7c4e5256d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:30.424598 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4291a-dc96-4965-80bc-5a7c4e5256d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:30.424865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4291a-dc96-4965-80bc-5a7c4e5256d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:30.425479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4291a-dc96-4965-80bc-5a7c4e5256d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:30.427405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4291a-dc96-4965-80bc-5a7c4e5256d3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:30.456526 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1172/2353] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:30:30 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:30:30.460826 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c47ab-9562-4f99-b5cd-e73bdb62aad9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:30.471379 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c47ab-9562-4f99-b5cd-e73bdb62aad9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:30.479875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:30.527765 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee1064-b380-4484-b718-c81da1766a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:30.528359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee1064-b380-4484-b718-c81da1766a07 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:30.528737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee1064-b380-4484-b718-c81da1766a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:30.529182 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee1064-b380-4484-b718-c81da1766a07 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:30.530275 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee1064-b380-4484-b718-c81da1766a07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:30.534419 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee1064-b380-4484-b718-c81da1766a07 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:30.555817 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1182/2354] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:30:30 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:30:30.559942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4291a-dc96-4965-80bc-5a7c4e5256d3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:30.566799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:30.570710 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4291a-dc96-4965-80bc-5a7c4e5256d3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:30.576350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a3ce2-ffc2-460f-bcc5-b017f11ab052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:30.577185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a3ce2-ffc2-460f-bcc5-b017f11ab052 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:30.577729 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a3ce2-ffc2-460f-bcc5-b017f11ab052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:30.578211 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a3ce2-ffc2-460f-bcc5-b017f11ab052 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:30.578906 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a3ce2-ffc2-460f-bcc5-b017f11ab052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:30.580604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a3ce2-ffc2-460f-bcc5-b017f11ab052 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:30.590927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1173/2355] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:30:30.595331 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee1064-b380-4484-b718-c81da1766a07 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:30.605159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee1064-b380-4484-b718-c81da1766a07 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:30.610410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ee336-dbd2-4859-a9c6-e97b51f79906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:30.610991 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ee336-dbd2-4859-a9c6-e97b51f79906 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:30.611264 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ee336-dbd2-4859-a9c6-e97b51f79906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:30.611531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ee336-dbd2-4859-a9c6-e97b51f79906 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:30.612157 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ee336-dbd2-4859-a9c6-e97b51f79906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:30.613717 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ee336-dbd2-4859-a9c6-e97b51f79906 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:30.624275 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1183/2356] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:30:30.628234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a3ce2-ffc2-460f-bcc5-b017f11ab052 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:30.638560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a3ce2-ffc2-460f-bcc5-b017f11ab052 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:30.643764 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f387f-4e60-4ecc-817a-0dc2b12071d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:30.644409 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f387f-4e60-4ecc-817a-0dc2b12071d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:30.644686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f387f-4e60-4ecc-817a-0dc2b12071d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:30.644958 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f387f-4e60-4ecc-817a-0dc2b12071d7 admin admin] PATH_INFO: `/v3/projects/ef9eec76edd24a338dc9fdd966fceacd/users/714f7c450ca945d4b2e149153ccfcdce/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:30.663738 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f387f-4e60-4ecc-817a-0dc2b12071d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef9eec76edd24a338dc9fdd966fceacd, user_id=714f7c450ca945d4b2e149153ccfcdce, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:30.666630 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f387f-4e60-4ecc-817a-0dc2b12071d7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:30.680361 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1174/2357] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:30:30 2022] PUT /identity/v3/projects/ef9eec76edd24a338dc9fdd966fceacd/users/714f7c450ca945d4b2e149153ccfcdce/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:30:30.685358 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207b7ed-bb5b-4a7b-8e40-b6a8a449d065 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:30.685676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207b7ed-bb5b-4a7b-8e40-b6a8a449d065 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:30.685979 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207b7ed-bb5b-4a7b-8e40-b6a8a449d065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:30.705761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8207b7ed-bb5b-4a7b-8e40-b6a8a449d065 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:30.707108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8207b7ed-bb5b-4a7b-8e40-b6a8a449d065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:30.771505 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1184/2358] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:30.781168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f387f-4e60-4ecc-817a-0dc2b12071d7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:30.783405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-252f387f-4e60-4ecc-817a-0dc2b12071d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:30.839523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f387f-4e60-4ecc-817a-0dc2b12071d7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:30.843164 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022fb531-c7e4-41cd-86ad-b7c08ca6c648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:30.843778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fb531-c7e4-41cd-86ad-b7c08ca6c648 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:30.844674 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fb531-c7e4-41cd-86ad-b7c08ca6c648 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:30.844961 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fb531-c7e4-41cd-86ad-b7c08ca6c648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:30.853696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022fb531-c7e4-41cd-86ad-b7c08ca6c648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:30.855166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022fb531-c7e4-41cd-86ad-b7c08ca6c648 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:30.865028 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1175/2359] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:31.790507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207b7ed-bb5b-4a7b-8e40-b6a8a449d065 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:31.793076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8207b7ed-bb5b-4a7b-8e40-b6a8a449d065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:31.808180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207b7ed-bb5b-4a7b-8e40-b6a8a449d065 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:31.813200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17247c-20b8-4545-95e6-9fd6cbfb54f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:31.813945 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17247c-20b8-4545-95e6-9fd6cbfb54f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:31.814368 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17247c-20b8-4545-95e6-9fd6cbfb54f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:31.815496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17247c-20b8-4545-95e6-9fd6cbfb54f0 admin admin] PATH_INFO: `/v3/users/bcb365c368b34f3d8444fca7abf5be4e` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:31.817036 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17247c-20b8-4545-95e6-9fd6cbfb54f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb365c368b34f3d8444fca7abf5be4e)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:31.819348 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17247c-20b8-4545-95e6-9fd6cbfb54f0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:31.877102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ad17247c-20b8-4545-95e6-9fd6cbfb54f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb365c368b34f3d8444fca7abf5be4e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:31.884881 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ad17247c-20b8-4545-95e6-9fd6cbfb54f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb365c368b34f3d8444fca7abf5be4e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:31.894305 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ad17247c-20b8-4545-95e6-9fd6cbfb54f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb365c368b34f3d8444fca7abf5be4e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:31.895010 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ad17247c-20b8-4545-95e6-9fd6cbfb54f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb365c368b34f3d8444fca7abf5be4e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:31.895661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ad17247c-20b8-4545-95e6-9fd6cbfb54f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb365c368b34f3d8444fca7abf5be4e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:31.920507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1185/2360] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:30:31 2022] DELETE /identity/v3/users/bcb365c368b34f3d8444fca7abf5be4e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:30:31.924995 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e17ac2-bb6a-4bf0-b6da-419b7c9b7de2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:31.926066 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e17ac2-bb6a-4bf0-b6da-419b7c9b7de2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:31.926573 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e17ac2-bb6a-4bf0-b6da-419b7c9b7de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:31.957422 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a2e17ac2-bb6a-4bf0-b6da-419b7c9b7de2 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:31.958815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a2e17ac2-bb6a-4bf0-b6da-419b7c9b7de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:32.040132 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1176/2361] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:30:32.050164 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17247c-20b8-4545-95e6-9fd6cbfb54f0 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:32.051728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ad17247c-20b8-4545-95e6-9fd6cbfb54f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:32.059590 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:32.130548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17247c-20b8-4545-95e6-9fd6cbfb54f0 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:32.139797 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81006f-d7f1-4115-91f0-a4f0d015482f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:32.142077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81006f-d7f1-4115-91f0-a4f0d015482f service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:32.142077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81006f-d7f1-4115-91f0-a4f0d015482f service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:32.142077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81006f-d7f1-4115-91f0-a4f0d015482f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:32.156143 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81006f-d7f1-4115-91f0-a4f0d015482f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:32.157663 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81006f-d7f1-4115-91f0-a4f0d015482f service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:32.172365 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1186/2362] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:32.555846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e17ac2-bb6a-4bf0-b6da-419b7c9b7de2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:32.557825 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a2e17ac2-bb6a-4bf0-b6da-419b7c9b7de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:32.576885 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e17ac2-bb6a-4bf0-b6da-419b7c9b7de2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:32.582303 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa6df7-3899-41ca-87d9-61f8f0060eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:32.583231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa6df7-3899-41ca-87d9-61f8f0060eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:32.583923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa6df7-3899-41ca-87d9-61f8f0060eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:32.584487 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa6df7-3899-41ca-87d9-61f8f0060eb3 admin admin] PATH_INFO: `/v3/projects/c1afb6d04ae94d06b6f82405907a62dd` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:32.586997 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa6df7-3899-41ca-87d9-61f8f0060eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1afb6d04ae94d06b6f82405907a62dd)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:32.589472 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa6df7-3899-41ca-87d9-61f8f0060eb3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:32.643774 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bbfa6df7-3899-41ca-87d9-61f8f0060eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1afb6d04ae94d06b6f82405907a62dd', 'request_id': 'req-bbfa6df7-3899-41ca-87d9-61f8f0060eb3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:32.650934 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bbfa6df7-3899-41ca-87d9-61f8f0060eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1afb6d04ae94d06b6f82405907a62dd', 'request_id': 'req-bbfa6df7-3899-41ca-87d9-61f8f0060eb3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:32.660645 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bbfa6df7-3899-41ca-87d9-61f8f0060eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1afb6d04ae94d06b6f82405907a62dd', 'request_id': 'req-bbfa6df7-3899-41ca-87d9-61f8f0060eb3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:32.661966 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:32.673210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bbfa6df7-3899-41ca-87d9-61f8f0060eb3 admin admin] The token cache is being invalidate because project c1afb6d04ae94d06b6f82405907a62dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:30:32.673590 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bbfa6df7-3899-41ca-87d9-61f8f0060eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:32.675004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bbfa6df7-3899-41ca-87d9-61f8f0060eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:32.682312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1177/2363] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:30:32 2022] DELETE /identity/v3/projects/c1afb6d04ae94d06b6f82405907a62dd => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:30:35.448585 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342440b-0e28-4bbf-b42d-fc598bbb6a6f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:35.449074 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342440b-0e28-4bbf-b42d-fc598bbb6a6f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:35.449179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342440b-0e28-4bbf-b42d-fc598bbb6a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:35.498918 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b342440b-0e28-4bbf-b42d-fc598bbb6a6f None None] MFA Rules not processed for user `714f7c450ca945d4b2e149153ccfcdce`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:35.502081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b342440b-0e28-4bbf-b42d-fc598bbb6a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:35.596153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1187/2364] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:30:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:35.600913 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa6df7-3899-41ca-87d9-61f8f0060eb3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:35.603844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bbfa6df7-3899-41ca-87d9-61f8f0060eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:35.725171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa6df7-3899-41ca-87d9-61f8f0060eb3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:35.731004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73edae-e454-495e-b93b-074294bcd1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:35.733212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73edae-e454-495e-b93b-074294bcd1aa admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:35.733580 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73edae-e454-495e-b93b-074294bcd1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:35.733854 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73edae-e454-495e-b93b-074294bcd1aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:35.735238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a73edae-e454-495e-b93b-074294bcd1aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:35.737137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a73edae-e454-495e-b93b-074294bcd1aa admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:35.774304 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1178/2365] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:30:35 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:30:35.775753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342440b-0e28-4bbf-b42d-fc598bbb6a6f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:35.788745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342440b-0e28-4bbf-b42d-fc598bbb6a6f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:35.867283 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253d0b1-ad9a-4862-bb70-bdffd0355515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:35.868037 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253d0b1-ad9a-4862-bb70-bdffd0355515 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:35.868425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253d0b1-ad9a-4862-bb70-bdffd0355515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:35.868800 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253d0b1-ad9a-4862-bb70-bdffd0355515 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:35.869541 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253d0b1-ad9a-4862-bb70-bdffd0355515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:35.871636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253d0b1-ad9a-4862-bb70-bdffd0355515 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:35.910434 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1188/2366] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:30:35 2022] POST /identity/v3/users => generated 298 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:30:35.911439 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73edae-e454-495e-b93b-074294bcd1aa admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:35.925309 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73edae-e454-495e-b93b-074294bcd1aa admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:35.931770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e05ca-41e6-464c-9167-4d60c5d19159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:35.932385 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e05ca-41e6-464c-9167-4d60c5d19159 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:35.933735 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e05ca-41e6-464c-9167-4d60c5d19159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:35.933735 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e05ca-41e6-464c-9167-4d60c5d19159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:35.933962 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e05ca-41e6-464c-9167-4d60c5d19159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:35.935569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e05ca-41e6-464c-9167-4d60c5d19159 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:35.948739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1179/2367] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:30:35.953028 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253d0b1-ad9a-4862-bb70-bdffd0355515 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:35.963984 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253d0b1-ad9a-4862-bb70-bdffd0355515 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:35.969981 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2e5a0-32e1-4c35-b2b9-de97f95f49a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:35.970704 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2e5a0-32e1-4c35-b2b9-de97f95f49a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:35.970982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2e5a0-32e1-4c35-b2b9-de97f95f49a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:35.971265 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2e5a0-32e1-4c35-b2b9-de97f95f49a8 admin admin] PATH_INFO: `/v3/projects/ba732da5f3fe45ccb54f20b4be5d366f/users/b9d9c312038e48dcb85f561e7feeb3b9/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:35.989408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2e5a0-32e1-4c35-b2b9-de97f95f49a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba732da5f3fe45ccb54f20b4be5d366f, user_id=b9d9c312038e48dcb85f561e7feeb3b9, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:35.993351 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2e5a0-32e1-4c35-b2b9-de97f95f49a8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:36.008602 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1189/2368] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:30:35 2022] PUT /identity/v3/projects/ba732da5f3fe45ccb54f20b4be5d366f/users/b9d9c312038e48dcb85f561e7feeb3b9/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:30:36.012877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e05ca-41e6-464c-9167-4d60c5d19159 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:36.025856 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e05ca-41e6-464c-9167-4d60c5d19159 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:36.124511 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173eed8-3b34-4558-9dcf-acf59db6ba76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:36.125194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173eed8-3b34-4558-9dcf-acf59db6ba76 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:36.125477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173eed8-3b34-4558-9dcf-acf59db6ba76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:36.125754 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173eed8-3b34-4558-9dcf-acf59db6ba76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:36.126658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1173eed8-3b34-4558-9dcf-acf59db6ba76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:36.128374 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1173eed8-3b34-4558-9dcf-acf59db6ba76 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:36.144542 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1180/2369] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:30:36.148430 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2e5a0-32e1-4c35-b2b9-de97f95f49a8 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:36.154168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:36.161243 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2e5a0-32e1-4c35-b2b9-de97f95f49a8 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:36.165910 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf39b85-09c4-4882-b962-044553a52b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:36.166761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf39b85-09c4-4882-b962-044553a52b19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:36.167550 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf39b85-09c4-4882-b962-044553a52b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:36.167838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf39b85-09c4-4882-b962-044553a52b19 admin admin] PATH_INFO: `/v3/projects/ba732da5f3fe45ccb54f20b4be5d366f/users/b9d9c312038e48dcb85f561e7feeb3b9/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:36.169903 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf39b85-09c4-4882-b962-044553a52b19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba732da5f3fe45ccb54f20b4be5d366f, user_id=b9d9c312038e48dcb85f561e7feeb3b9, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:36.173680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf39b85-09c4-4882-b962-044553a52b19 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:36.194770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1190/2370] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:30:36 2022] PUT /identity/v3/projects/ba732da5f3fe45ccb54f20b4be5d366f/users/b9d9c312038e48dcb85f561e7feeb3b9/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:30:41.937999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7a035-f06f-421c-9840-07a3dd956bf5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:41.938561 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7a035-f06f-421c-9840-07a3dd956bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:41.938561 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7a035-f06f-421c-9840-07a3dd956bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:41.972465 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-41a7a035-f06f-421c-9840-07a3dd956bf5 None None] MFA Rules not processed for user `b9d9c312038e48dcb85f561e7feeb3b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:41.973702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-41a7a035-f06f-421c-9840-07a3dd956bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:42.038999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:42.061129 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1181/2371] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:30:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:42.076094 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf39b85-09c4-4882-b962-044553a52b19 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:42.079407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9bf39b85-09c4-4882-b962-044553a52b19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:42.150450 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf39b85-09c4-4882-b962-044553a52b19 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:42.154783 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4de62-3f92-4203-adf2-99eaf374e7b6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7776a6a1faf34b369e399d83bb3e18c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:42.155473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4de62-3f92-4203-adf2-99eaf374e7b6 service glance] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:42.155895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4de62-3f92-4203-adf2-99eaf374e7b6 service glance] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:42.156641 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4de62-3f92-4203-adf2-99eaf374e7b6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:42.229338 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed4de62-3f92-4203-adf2-99eaf374e7b6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:42.230965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed4de62-3f92-4203-adf2-99eaf374e7b6 service glance] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:42.242862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1191/2372] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:42.418959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a50887-f890-4649-9c9c-2a44457aa5e7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:42.419244 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a50887-f890-4649-9c9c-2a44457aa5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:42.419512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a50887-f890-4649-9c9c-2a44457aa5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:42.456706 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-09a50887-f890-4649-9c9c-2a44457aa5e7 None None] MFA Rules not processed for user `b88ccd3e5c4c48d3a27f4eaa7874e28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:42.458413 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-09a50887-f890-4649-9c9c-2a44457aa5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:42.557089 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1182/2373] 10.222.0.39 () {58 vars in 1009 bytes} [Mon Aug 8 02:30:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:42.615127 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4de62-3f92-4203-adf2-99eaf374e7b6 service glance] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:42.617422 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bed4de62-3f92-4203-adf2-99eaf374e7b6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:42.731927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4de62-3f92-4203-adf2-99eaf374e7b6 service glance] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:42.737903 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee85960-5cf0-4041-98b5-162834ee18f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac60b4d621234225830c94edfac400e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:42.738678 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee85960-5cf0-4041-98b5-162834ee18f6 service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:42.738989 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee85960-5cf0-4041-98b5-162834ee18f6 service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:42.739334 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee85960-5cf0-4041-98b5-162834ee18f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:42.748751 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee85960-5cf0-4041-98b5-162834ee18f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:42.751230 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee85960-5cf0-4041-98b5-162834ee18f6 service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:42.764208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1192/2374] 10.222.0.39 () {60 vars in 1384 bytes} [Mon Aug 8 02:30:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:30:51.986754 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875882d-d438-4cea-8f61-5368fc36f3b1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:51.987278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875882d-d438-4cea-8f61-5368fc36f3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:51.987278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875882d-d438-4cea-8f61-5368fc36f3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:52.006155 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d875882d-d438-4cea-8f61-5368fc36f3b1 None None] MFA Rules not processed for user `714f7c450ca945d4b2e149153ccfcdce`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:52.007375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d875882d-d438-4cea-8f61-5368fc36f3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:52.014138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1183/2375] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:30:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:52.270773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee85960-5cf0-4041-98b5-162834ee18f6 service swift] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:52.272257 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cee85960-5cf0-4041-98b5-162834ee18f6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:52.308025 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:52.331208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee85960-5cf0-4041-98b5-162834ee18f6 service swift] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:52.334784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c14665-2a0d-4afa-92fa-35633ccb4ff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:52.335489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c14665-2a0d-4afa-92fa-35633ccb4ff5 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:52.335646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c14665-2a0d-4afa-92fa-35633ccb4ff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:52.335920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c14665-2a0d-4afa-92fa-35633ccb4ff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:52.337643 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e7c14665-2a0d-4afa-92fa-35633ccb4ff5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:52.348228 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c14665-2a0d-4afa-92fa-35633ccb4ff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:52.349649 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c14665-2a0d-4afa-92fa-35633ccb4ff5 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:52.357746 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1193/2376] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:52.668619 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875882d-d438-4cea-8f61-5368fc36f3b1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:52.670108 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d875882d-d438-4cea-8f61-5368fc36f3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:52.741091 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875882d-d438-4cea-8f61-5368fc36f3b1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:52.746007 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce82fc-e74f-4873-ad7c-daa14a2eed7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f62745b92a864480a9438de581e1d05e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:52.746634 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce82fc-e74f-4873-ad7c-daa14a2eed7a service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:52.746909 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce82fc-e74f-4873-ad7c-daa14a2eed7a service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:52.747174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce82fc-e74f-4873-ad7c-daa14a2eed7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:52.748844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-63ce82fc-e74f-4873-ad7c-daa14a2eed7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:52.759818 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce82fc-e74f-4873-ad7c-daa14a2eed7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:52.761375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce82fc-e74f-4873-ad7c-daa14a2eed7a service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:52.771490 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1184/2377] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:54.847839 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5c922-6160-4b5d-88f4-1f97d4bbb2be None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:54.848385 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5c922-6160-4b5d-88f4-1f97d4bbb2be None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:54.848385 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5c922-6160-4b5d-88f4-1f97d4bbb2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:54.869508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8dd5c922-6160-4b5d-88f4-1f97d4bbb2be None None] MFA Rules not processed for user `b88ccd3e5c4c48d3a27f4eaa7874e28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:54.870773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8dd5c922-6160-4b5d-88f4-1f97d4bbb2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:54.879044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1194/2378] 10.222.0.39 () {58 vars in 1009 bytes} [Mon Aug 8 02:30:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:54.891447 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce82fc-e74f-4873-ad7c-daa14a2eed7a service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:54.902886 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce82fc-e74f-4873-ad7c-daa14a2eed7a service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:54.908015 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6746f-27c7-4149-9468-c624d252fc5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac60b4d621234225830c94edfac400e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:54.908610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6746f-27c7-4149-9468-c624d252fc5c service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:54.908906 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6746f-27c7-4149-9468-c624d252fc5c service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:54.909160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6746f-27c7-4149-9468-c624d252fc5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:54.917541 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6746f-27c7-4149-9468-c624d252fc5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:54.919162 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6746f-27c7-4149-9468-c624d252fc5c service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:54.929460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1185/2379] 10.222.0.39 () {60 vars in 1384 bytes} [Mon Aug 8 02:30:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:31:04.864171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=14 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:04.925789 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=70 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:15.806698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5c922-6160-4b5d-88f4-1f97d4bbb2be None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:15.808595 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8dd5c922-6160-4b5d-88f4-1f97d4bbb2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:15.822585 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5c922-6160-4b5d-88f4-1f97d4bbb2be None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:15.826130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c8b54-f054-49fc-9886-146d942561d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:15.826758 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c8b54-f054-49fc-9886-146d942561d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:15.827037 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c8b54-f054-49fc-9886-146d942561d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:15.827308 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c8b54-f054-49fc-9886-146d942561d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:15.828983 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-662c8b54-f054-49fc-9886-146d942561d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:15.839010 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c8b54-f054-49fc-9886-146d942561d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:15.840413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c8b54-f054-49fc-9886-146d942561d9 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:15.849689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1195/2380] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:16.414835 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6746f-27c7-4149-9468-c624d252fc5c service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:16.416355 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a1c6746f-27c7-4149-9468-c624d252fc5c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:16.491687 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6746f-27c7-4149-9468-c624d252fc5c service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:16.497203 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c16658-e8e9-4361-9df7-0113d0ab60ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:16.497805 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c16658-e8e9-4361-9df7-0113d0ab60ba service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:16.498146 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c16658-e8e9-4361-9df7-0113d0ab60ba service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:16.498448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c16658-e8e9-4361-9df7-0113d0ab60ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:16.500177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-18c16658-e8e9-4361-9df7-0113d0ab60ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:16.513070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c16658-e8e9-4361-9df7-0113d0ab60ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:16.514692 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c16658-e8e9-4361-9df7-0113d0ab60ba service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:16.525357 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1186/2381] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:23.476027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c8b54-f054-49fc-9886-146d942561d9 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:23.490661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c8b54-f054-49fc-9886-146d942561d9 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:23.496363 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cfe07d-5183-4ed9-be57-74f1399cb58a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:23.496941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfe07d-5183-4ed9-be57-74f1399cb58a service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:23.497397 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfe07d-5183-4ed9-be57-74f1399cb58a service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:23.497669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfe07d-5183-4ed9-be57-74f1399cb58a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:23.499449 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-34cfe07d-5183-4ed9-be57-74f1399cb58a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:23.511906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cfe07d-5183-4ed9-be57-74f1399cb58a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:23.513658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cfe07d-5183-4ed9-be57-74f1399cb58a service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:23.524554 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1196/2382] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:26.520916 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=34 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:33.525168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=12 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:41.003703 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c16658-e8e9-4361-9df7-0113d0ab60ba service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:41.012861 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c16658-e8e9-4361-9df7-0113d0ab60ba service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:41.016378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec20ea2-174a-4037-9142-d1b52fb30106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:41.016966 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec20ea2-174a-4037-9142-d1b52fb30106 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:41.017244 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec20ea2-174a-4037-9142-d1b52fb30106 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:41.018029 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec20ea2-174a-4037-9142-d1b52fb30106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:41.026730 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec20ea2-174a-4037-9142-d1b52fb30106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:41.028149 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec20ea2-174a-4037-9142-d1b52fb30106 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:41.037972 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1187/2383] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:42.607131 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cfe07d-5183-4ed9-be57-74f1399cb58a service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:42.619314 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cfe07d-5183-4ed9-be57-74f1399cb58a service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:42.626665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92628c9-929e-4570-a024-6f8d3c35a0e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:42.627844 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92628c9-929e-4570-a024-6f8d3c35a0e8 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:42.628357 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92628c9-929e-4570-a024-6f8d3c35a0e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:42.628662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92628c9-929e-4570-a024-6f8d3c35a0e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:42.630504 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e92628c9-929e-4570-a024-6f8d3c35a0e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:42.644628 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92628c9-929e-4570-a024-6f8d3c35a0e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:42.646526 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92628c9-929e-4570-a024-6f8d3c35a0e8 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:42.658629 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1197/2384] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:50.103663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec20ea2-174a-4037-9142-d1b52fb30106 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:50.115478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec20ea2-174a-4037-9142-d1b52fb30106 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:50.120400 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369e2c1-eb85-4877-abfa-d68a0ab3c545 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f62745b92a864480a9438de581e1d05e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:50.120990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369e2c1-eb85-4877-abfa-d68a0ab3c545 service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:50.121267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369e2c1-eb85-4877-abfa-d68a0ab3c545 service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:50.121538 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369e2c1-eb85-4877-abfa-d68a0ab3c545 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:50.123267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c369e2c1-eb85-4877-abfa-d68a0ab3c545 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:50.135924 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369e2c1-eb85-4877-abfa-d68a0ab3c545 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:50.137600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369e2c1-eb85-4877-abfa-d68a0ab3c545 service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:50.150213 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1188/2385] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:50.158203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92628c9-929e-4570-a024-6f8d3c35a0e8 service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:50.159827 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e92628c9-929e-4570-a024-6f8d3c35a0e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:50.227982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92628c9-929e-4570-a024-6f8d3c35a0e8 service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:50.232595 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea32c72-3ffc-4369-9128-64defdf7616c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:50.233217 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea32c72-3ffc-4369-9128-64defdf7616c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:50.233495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea32c72-3ffc-4369-9128-64defdf7616c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:50.233765 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea32c72-3ffc-4369-9128-64defdf7616c admin admin] PATH_INFO: `/v3/users/ef2ca1a30a76468cbe02b20efd6ca37e` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:50.234886 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea32c72-3ffc-4369-9128-64defdf7616c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef2ca1a30a76468cbe02b20efd6ca37e)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:50.237339 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea32c72-3ffc-4369-9128-64defdf7616c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:50.280767 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9ea32c72-3ffc-4369-9128-64defdf7616c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef2ca1a30a76468cbe02b20efd6ca37e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:50.288045 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9ea32c72-3ffc-4369-9128-64defdf7616c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef2ca1a30a76468cbe02b20efd6ca37e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:50.295004 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9ea32c72-3ffc-4369-9128-64defdf7616c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef2ca1a30a76468cbe02b20efd6ca37e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:50.295636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9ea32c72-3ffc-4369-9128-64defdf7616c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef2ca1a30a76468cbe02b20efd6ca37e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:50.296234 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9ea32c72-3ffc-4369-9128-64defdf7616c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef2ca1a30a76468cbe02b20efd6ca37e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:50.313570 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1198/2386] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:31:50 2022] DELETE /identity/v3/users/ef2ca1a30a76468cbe02b20efd6ca37e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:50.490599 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369e2c1-eb85-4877-abfa-d68a0ab3c545 service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:50.501664 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369e2c1-eb85-4877-abfa-d68a0ab3c545 service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:50.558927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b8cd5-a48b-4188-97ad-9cfc40d8e727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:50.559472 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b8cd5-a48b-4188-97ad-9cfc40d8e727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:50.559743 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b8cd5-a48b-4188-97ad-9cfc40d8e727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:50.560009 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b8cd5-a48b-4188-97ad-9cfc40d8e727 admin admin] PATH_INFO: `/v3/projects/2d75f4c0fe9148e7bf633a79dbfd6bb9` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:50.561097 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b8cd5-a48b-4188-97ad-9cfc40d8e727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d75f4c0fe9148e7bf633a79dbfd6bb9)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:50.563206 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b8cd5-a48b-4188-97ad-9cfc40d8e727 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:50.609161 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bd7b8cd5-a48b-4188-97ad-9cfc40d8e727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d75f4c0fe9148e7bf633a79dbfd6bb9', 'request_id': 'req-bd7b8cd5-a48b-4188-97ad-9cfc40d8e727'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:50.615175 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bd7b8cd5-a48b-4188-97ad-9cfc40d8e727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d75f4c0fe9148e7bf633a79dbfd6bb9', 'request_id': 'req-bd7b8cd5-a48b-4188-97ad-9cfc40d8e727'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:50.622470 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bd7b8cd5-a48b-4188-97ad-9cfc40d8e727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d75f4c0fe9148e7bf633a79dbfd6bb9', 'request_id': 'req-bd7b8cd5-a48b-4188-97ad-9cfc40d8e727'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:50.633370 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bd7b8cd5-a48b-4188-97ad-9cfc40d8e727 admin admin] The token cache is being invalidate because project 2d75f4c0fe9148e7bf633a79dbfd6bb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:31:50.633661 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bd7b8cd5-a48b-4188-97ad-9cfc40d8e727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:50.634311 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bd7b8cd5-a48b-4188-97ad-9cfc40d8e727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:50.635790 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1189/2387] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:31:50 2022] DELETE /identity/v3/projects/2d75f4c0fe9148e7bf633a79dbfd6bb9 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:52.981279 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea32c72-3ffc-4369-9128-64defdf7616c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:52.982745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9ea32c72-3ffc-4369-9128-64defdf7616c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:53.050946 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:53.055484 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea32c72-3ffc-4369-9128-64defdf7616c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:53.118024 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34957a3-451d-4322-ba89-299f0b159edd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:53.118662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34957a3-451d-4322-ba89-299f0b159edd service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:53.118940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34957a3-451d-4322-ba89-299f0b159edd service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:53.119206 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34957a3-451d-4322-ba89-299f0b159edd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:53.121033 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d34957a3-451d-4322-ba89-299f0b159edd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:53.176700 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34957a3-451d-4322-ba89-299f0b159edd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:53.178356 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34957a3-451d-4322-ba89-299f0b159edd service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:53.187303 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1199/2388] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:00.630469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=63,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:03.183200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=54 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:03.675740 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b8cd5-a48b-4188-97ad-9cfc40d8e727 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:03.677162 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bd7b8cd5-a48b-4188-97ad-9cfc40d8e727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:03.693662 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b8cd5-a48b-4188-97ad-9cfc40d8e727 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:03.699178 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a173ee1-f3d8-4f55-bf08-830eeb8af4cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:03.699716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a173ee1-f3d8-4f55-bf08-830eeb8af4cf service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:03.699990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a173ee1-f3d8-4f55-bf08-830eeb8af4cf service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:03.700258 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a173ee1-f3d8-4f55-bf08-830eeb8af4cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:03.701961 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3a173ee1-f3d8-4f55-bf08-830eeb8af4cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:03.760445 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a173ee1-f3d8-4f55-bf08-830eeb8af4cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:03.762066 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a173ee1-f3d8-4f55-bf08-830eeb8af4cf service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:03.771171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1190/2389] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:13.767007 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=28 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:21.305044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bf40d-f0d7-4f05-840d-360fef733388 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:21.305633 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bf40d-f0d7-4f05-840d-360fef733388 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:21.305633 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bf40d-f0d7-4f05-840d-360fef733388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:21.328332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a31bf40d-f0d7-4f05-840d-360fef733388 None None] MFA Rules not processed for user `1fd1b40cbd884915bdfac150f9718e32`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:21.329567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a31bf40d-f0d7-4f05-840d-360fef733388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:21.338332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1200/2390] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:32:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:27.952075 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a173ee1-f3d8-4f55-bf08-830eeb8af4cf service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:27.964712 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a173ee1-f3d8-4f55-bf08-830eeb8af4cf service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:27.972483 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b94c3-575c-42f3-bfab-7140b200c7bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:27.974018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b94c3-575c-42f3-bfab-7140b200c7bb service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:27.974528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b94c3-575c-42f3-bfab-7140b200c7bb service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:27.974996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b94c3-575c-42f3-bfab-7140b200c7bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:27.977959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a20b94c3-575c-42f3-bfab-7140b200c7bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:28.044131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b94c3-575c-42f3-bfab-7140b200c7bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:28.046116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b94c3-575c-42f3-bfab-7140b200c7bb service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:28.057354 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1191/2391] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:31.320836 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:38.051441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=28 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:41.665922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3e372-aa21-4d70-9032-98774767379a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:41.666483 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3e372-aa21-4d70-9032-98774767379a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:41.666483 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3e372-aa21-4d70-9032-98774767379a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:41.685202 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-14c3e372-aa21-4d70-9032-98774767379a None None] MFA Rules not processed for user `43dd72b161f8442285dea411741d66b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:41.686429 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-14c3e372-aa21-4d70-9032-98774767379a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:41.694462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1201/2392] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:32:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:41.996870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b94c3-575c-42f3-bfab-7140b200c7bb service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:41.998290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a20b94c3-575c-42f3-bfab-7140b200c7bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:42.054261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b94c3-575c-42f3-bfab-7140b200c7bb service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:42.057942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefcc971-64de-400b-8f7b-283ebfccea3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:42.058577 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcc971-64de-400b-8f7b-283ebfccea3b service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:42.058858 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcc971-64de-400b-8f7b-283ebfccea3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:42.059129 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcc971-64de-400b-8f7b-283ebfccea3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:42.060874 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cefcc971-64de-400b-8f7b-283ebfccea3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:42.071596 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefcc971-64de-400b-8f7b-283ebfccea3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:42.073069 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefcc971-64de-400b-8f7b-283ebfccea3b service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:42.081315 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1192/2393] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:51.680123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:52.077558 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=26 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:42.902300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e20ca-65a5-48b9-a509-7f0e6bc2fd92 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:42.902835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e20ca-65a5-48b9-a509-7f0e6bc2fd92 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:42.902835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e20ca-65a5-48b9-a509-7f0e6bc2fd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:42.936829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-cf4e20ca-65a5-48b9-a509-7f0e6bc2fd92 None None] MFA Rules not processed for user `17286ff2e1764ec7b91e6274c7162437`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:42.938352 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cf4e20ca-65a5-48b9-a509-7f0e6bc2fd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:42.962968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1202/2394] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:33:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:52.541678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefcc971-64de-400b-8f7b-283ebfccea3b service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:52.543904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cefcc971-64de-400b-8f7b-283ebfccea3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:52.614737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefcc971-64de-400b-8f7b-283ebfccea3b service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:52.617686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c8b1a-3f5c-4e1a-8679-534256766b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:52.618338 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c8b1a-3f5c-4e1a-8679-534256766b8f service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:52.618621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c8b1a-3f5c-4e1a-8679-534256766b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:52.618895 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c8b1a-3f5c-4e1a-8679-534256766b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:52.620605 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-197c8b1a-3f5c-4e1a-8679-534256766b8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:52.710565 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c8b1a-3f5c-4e1a-8679-534256766b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:52.712000 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c8b1a-3f5c-4e1a-8679-534256766b8f service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:52.730459 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1193/2395] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:52.960935 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=15 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:56.075289 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e20ca-65a5-48b9-a509-7f0e6bc2fd92 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:56.076756 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cf4e20ca-65a5-48b9-a509-7f0e6bc2fd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:56.090541 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e20ca-65a5-48b9-a509-7f0e6bc2fd92 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:56.095086 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80f5c0-b73f-437c-a180-36a21c83a626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:56.095703 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80f5c0-b73f-437c-a180-36a21c83a626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:56.095980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80f5c0-b73f-437c-a180-36a21c83a626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:56.096276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80f5c0-b73f-437c-a180-36a21c83a626 admin admin] PATH_INFO: `/v3/users/1fd1b40cbd884915bdfac150f9718e32` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:56.097387 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80f5c0-b73f-437c-a180-36a21c83a626 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fd1b40cbd884915bdfac150f9718e32)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:56.099341 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80f5c0-b73f-437c-a180-36a21c83a626 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:56.149040 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bf80f5c0-b73f-437c-a180-36a21c83a626 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fd1b40cbd884915bdfac150f9718e32'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:56.156080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bf80f5c0-b73f-437c-a180-36a21c83a626 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fd1b40cbd884915bdfac150f9718e32'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:56.163842 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bf80f5c0-b73f-437c-a180-36a21c83a626 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fd1b40cbd884915bdfac150f9718e32'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:56.164468 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bf80f5c0-b73f-437c-a180-36a21c83a626 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fd1b40cbd884915bdfac150f9718e32'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:56.165082 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bf80f5c0-b73f-437c-a180-36a21c83a626 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fd1b40cbd884915bdfac150f9718e32'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:56.182286 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1203/2396] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:56 2022] DELETE /identity/v3/users/1fd1b40cbd884915bdfac150f9718e32 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:56.359835 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c8b1a-3f5c-4e1a-8679-534256766b8f service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:56.371193 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c8b1a-3f5c-4e1a-8679-534256766b8f service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:56.427746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c362e8-9007-42ab-933b-a61380666d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:56.428378 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c362e8-9007-42ab-933b-a61380666d88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:56.428660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c362e8-9007-42ab-933b-a61380666d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:56.428933 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c362e8-9007-42ab-933b-a61380666d88 admin admin] PATH_INFO: `/v3/projects/12bf9f0d74de4a57918b688a6d969b28` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:56.430055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c362e8-9007-42ab-933b-a61380666d88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12bf9f0d74de4a57918b688a6d969b28)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:56.432013 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c362e8-9007-42ab-933b-a61380666d88 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:56.443570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:56.478462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d4c362e8-9007-42ab-933b-a61380666d88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12bf9f0d74de4a57918b688a6d969b28', 'request_id': 'req-d4c362e8-9007-42ab-933b-a61380666d88'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:56.483915 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d4c362e8-9007-42ab-933b-a61380666d88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12bf9f0d74de4a57918b688a6d969b28', 'request_id': 'req-d4c362e8-9007-42ab-933b-a61380666d88'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:56.492345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d4c362e8-9007-42ab-933b-a61380666d88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12bf9f0d74de4a57918b688a6d969b28', 'request_id': 'req-d4c362e8-9007-42ab-933b-a61380666d88'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:56.503333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d4c362e8-9007-42ab-933b-a61380666d88 admin admin] The token cache is being invalidate because project 12bf9f0d74de4a57918b688a6d969b28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:56.503615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d4c362e8-9007-42ab-933b-a61380666d88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:56.504233 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d4c362e8-9007-42ab-933b-a61380666d88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:56.505687 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1194/2397] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:56 2022] DELETE /identity/v3/projects/12bf9f0d74de4a57918b688a6d969b28 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:56.510549 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb8d8f-1a12-48e0-ba60-22e01d458979 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:56.510839 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb8d8f-1a12-48e0-ba60-22e01d458979 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:56.511110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb8d8f-1a12-48e0-ba60-22e01d458979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:56.570367 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-93cb8d8f-1a12-48e0-ba60-22e01d458979 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:56.571681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-93cb8d8f-1a12-48e0-ba60-22e01d458979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:56.631415 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1204/2398] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:56.639200 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e599227-cfcc-4388-af67-eba83c492cf9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:56.639488 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e599227-cfcc-4388-af67-eba83c492cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:56.639763 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e599227-cfcc-4388-af67-eba83c492cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:56.660803 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3e599227-cfcc-4388-af67-eba83c492cf9 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:56.662020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3e599227-cfcc-4388-af67-eba83c492cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:56.670547 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1195/2399] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:56.674769 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb8d8f-1a12-48e0-ba60-22e01d458979 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:56.684420 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb8d8f-1a12-48e0-ba60-22e01d458979 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:56.688992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185fb68-6d0b-4ab1-b4d8-b9f065873c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:56.689577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185fb68-6d0b-4ab1-b4d8-b9f065873c0e admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:56.690100 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185fb68-6d0b-4ab1-b4d8-b9f065873c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:56.690445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185fb68-6d0b-4ab1-b4d8-b9f065873c0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:56.691120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185fb68-6d0b-4ab1-b4d8-b9f065873c0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:33:56.691424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185fb68-6d0b-4ab1-b4d8-b9f065873c0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:56.693015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185fb68-6d0b-4ab1-b4d8-b9f065873c0e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:56.704613 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1205/2400] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:33:56.709367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e599227-cfcc-4388-af67-eba83c492cf9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:56.718747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e599227-cfcc-4388-af67-eba83c492cf9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:56.724290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f51fb4-7b02-4cbb-ac38-85119676ab35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:56.724904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f51fb4-7b02-4cbb-ac38-85119676ab35 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:56.725177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f51fb4-7b02-4cbb-ac38-85119676ab35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:56.725439 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f51fb4-7b02-4cbb-ac38-85119676ab35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:56.726066 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f51fb4-7b02-4cbb-ac38-85119676ab35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:56.727979 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f51fb4-7b02-4cbb-ac38-85119676ab35 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:56.758867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1196/2401] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:56 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:56.759992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185fb68-6d0b-4ab1-b4d8-b9f065873c0e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:56.770679 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185fb68-6d0b-4ab1-b4d8-b9f065873c0e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:56.799747 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:56.834812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ad223-8141-4256-9198-7ce613acb9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:56.835334 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ad223-8141-4256-9198-7ce613acb9ac admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:56.835616 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ad223-8141-4256-9198-7ce613acb9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:56.835889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ad223-8141-4256-9198-7ce613acb9ac admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:56.836512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ad223-8141-4256-9198-7ce613acb9ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:56.838447 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ad223-8141-4256-9198-7ce613acb9ac admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:56.861027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1206/2402] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:56 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:33:56.865225 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f51fb4-7b02-4cbb-ac38-85119676ab35 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:56.875642 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f51fb4-7b02-4cbb-ac38-85119676ab35 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:56.880724 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b85c37-59e2-49fe-bcdf-79fd173d4c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:56.881308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b85c37-59e2-49fe-bcdf-79fd173d4c4f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:56.881583 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b85c37-59e2-49fe-bcdf-79fd173d4c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:56.881854 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b85c37-59e2-49fe-bcdf-79fd173d4c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:56.882522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b85c37-59e2-49fe-bcdf-79fd173d4c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:56.884117 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b85c37-59e2-49fe-bcdf-79fd173d4c4f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:56.894741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1197/2403] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:56.898695 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ad223-8141-4256-9198-7ce613acb9ac admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:56.908477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ad223-8141-4256-9198-7ce613acb9ac admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:56.912999 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5615f-769f-4066-8c90-784aca85311a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:56.913580 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5615f-769f-4066-8c90-784aca85311a admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:56.913857 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5615f-769f-4066-8c90-784aca85311a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:56.914124 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5615f-769f-4066-8c90-784aca85311a admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:56.914796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5615f-769f-4066-8c90-784aca85311a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:56.916368 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5615f-769f-4066-8c90-784aca85311a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:56.926732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1207/2404] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:56.930843 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b85c37-59e2-49fe-bcdf-79fd173d4c4f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:56.941562 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b85c37-59e2-49fe-bcdf-79fd173d4c4f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:56.946182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78464784-01ac-451d-b4b2-55790fb3126f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:56.946930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78464784-01ac-451d-b4b2-55790fb3126f admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:56.947134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78464784-01ac-451d-b4b2-55790fb3126f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:56.947467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78464784-01ac-451d-b4b2-55790fb3126f admin admin] PATH_INFO: `/v3/projects/6dcedd52439b47689047813e0566ba78/users/5de4e28bfee14bae93c7f7a8f3f098fc/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:56.962488 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78464784-01ac-451d-b4b2-55790fb3126f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dcedd52439b47689047813e0566ba78, user_id=5de4e28bfee14bae93c7f7a8f3f098fc, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:56.965213 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78464784-01ac-451d-b4b2-55790fb3126f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:56.977306 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1198/2405] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:56 2022] PUT /identity/v3/projects/6dcedd52439b47689047813e0566ba78/users/5de4e28bfee14bae93c7f7a8f3f098fc/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:56.982513 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f78e6-733e-46e9-8588-03f975e43d72 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:56.982801 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f78e6-733e-46e9-8588-03f975e43d72 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:56.983075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f78e6-733e-46e9-8588-03f975e43d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:57.001577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b94f78e6-733e-46e9-8588-03f975e43d72 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:57.003310 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b94f78e6-733e-46e9-8588-03f975e43d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:57.061072 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1208/2406] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:57.071374 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78464784-01ac-451d-b4b2-55790fb3126f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:57.073108 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-78464784-01ac-451d-b4b2-55790fb3126f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:57.129013 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78464784-01ac-451d-b4b2-55790fb3126f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:57.132540 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c45ae1-ba8e-4f31-b4de-7021cdbc0d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:57.133131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c45ae1-ba8e-4f31-b4de-7021cdbc0d2e service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:57.133410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c45ae1-ba8e-4f31-b4de-7021cdbc0d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:57.133682 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c45ae1-ba8e-4f31-b4de-7021cdbc0d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:57.142651 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c45ae1-ba8e-4f31-b4de-7021cdbc0d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:57.144699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c45ae1-ba8e-4f31-b4de-7021cdbc0d2e service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:57.154022 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1199/2407] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:01.571547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d273b-949c-437e-885e-35fe4e4d6150 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:01.572069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d273b-949c-437e-885e-35fe4e4d6150 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:01.572069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d273b-949c-437e-885e-35fe4e4d6150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:01.604413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-497d273b-949c-437e-885e-35fe4e4d6150 None None] MFA Rules not processed for user `5de4e28bfee14bae93c7f7a8f3f098fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:01.605722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-497d273b-949c-437e-885e-35fe4e4d6150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:01.681674 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1209/2408] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:34:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:01.697540 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c45ae1-ba8e-4f31-b4de-7021cdbc0d2e service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:01.699222 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-96c45ae1-ba8e-4f31-b4de-7021cdbc0d2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:01.750200 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:01.821950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c45ae1-ba8e-4f31-b4de-7021cdbc0d2e service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:01.828332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b978ab-18de-4b3c-b416-abed2a77b95c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:01.828956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b978ab-18de-4b3c-b416-abed2a77b95c service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:01.829231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b978ab-18de-4b3c-b416-abed2a77b95c service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:01.829499 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b978ab-18de-4b3c-b416-abed2a77b95c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:01.837873 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b978ab-18de-4b3c-b416-abed2a77b95c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:01.839510 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b978ab-18de-4b3c-b416-abed2a77b95c service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:01.849592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1200/2409] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:01 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:03.150895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253e10c-c485-4aae-91a3-b87a10510aaf None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:03.151423 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253e10c-c485-4aae-91a3-b87a10510aaf None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:03.151423 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253e10c-c485-4aae-91a3-b87a10510aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:03.171234 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-4253e10c-c485-4aae-91a3-b87a10510aaf None None] MFA Rules not processed for user `5de4e28bfee14bae93c7f7a8f3f098fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:03.172381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4253e10c-c485-4aae-91a3-b87a10510aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:03.179862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1210/2410] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:34:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:11.845212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=28 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:13.165933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:24.151638 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab5bf6-80ee-44f2-838f-002052a6a5e5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:24.152082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab5bf6-80ee-44f2-838f-002052a6a5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:24.152193 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab5bf6-80ee-44f2-838f-002052a6a5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:24.181552 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e0ab5bf6-80ee-44f2-838f-002052a6a5e5 None None] MFA Rules not processed for user `b88ccd3e5c4c48d3a27f4eaa7874e28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:24.182828 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e0ab5bf6-80ee-44f2-838f-002052a6a5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:24.239389 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1201/2411] 10.222.0.39 () {58 vars in 967 bytes} [Mon Aug 8 02:34:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:24.278739 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253e10c-c485-4aae-91a3-b87a10510aaf None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:24.280212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4253e10c-c485-4aae-91a3-b87a10510aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:24.354996 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253e10c-c485-4aae-91a3-b87a10510aaf None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:24.365759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea95f3a-585a-4583-a050-f30261dafee3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac60b4d621234225830c94edfac400e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:24.366446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea95f3a-585a-4583-a050-f30261dafee3 service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:24.366730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea95f3a-585a-4583-a050-f30261dafee3 service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:24.367001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea95f3a-585a-4583-a050-f30261dafee3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:24.375795 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea95f3a-585a-4583-a050-f30261dafee3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:24.377359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea95f3a-585a-4583-a050-f30261dafee3 service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:24.385807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1211/2412] 10.222.0.39 () {60 vars in 1384 bytes} [Mon Aug 8 02:34:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:34:31.109452 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab5bf6-80ee-44f2-838f-002052a6a5e5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:31.110998 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e0ab5bf6-80ee-44f2-838f-002052a6a5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:31.193564 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea95f3a-585a-4583-a050-f30261dafee3 service swift] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:31.195579 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3ea95f3a-585a-4583-a050-f30261dafee3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:31.218170 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea95f3a-585a-4583-a050-f30261dafee3 service swift] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:31.220322 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab5bf6-80ee-44f2-838f-002052a6a5e5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:31.224549 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a991be1-c4f1-4bdf-b4e7-50efdf8f0a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:31.225743 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a991be1-c4f1-4bdf-b4e7-50efdf8f0a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:31.226319 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a991be1-c4f1-4bdf-b4e7-50efdf8f0a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:31.226504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd2613-4d9a-46cd-ab69-8dfd9c9748ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f62745b92a864480a9438de581e1d05e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:31.227033 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a991be1-c4f1-4bdf-b4e7-50efdf8f0a85 admin admin] PATH_INFO: `/v3/users/714f7c450ca945d4b2e149153ccfcdce` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:31.227147 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd2613-4d9a-46cd-ab69-8dfd9c9748ac service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:31.227384 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd2613-4d9a-46cd-ab69-8dfd9c9748ac service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:31.227654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd2613-4d9a-46cd-ab69-8dfd9c9748ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:31.228728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a991be1-c4f1-4bdf-b4e7-50efdf8f0a85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=714f7c450ca945d4b2e149153ccfcdce)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:31.229394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9afd2613-4d9a-46cd-ab69-8dfd9c9748ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:31.231016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a991be1-c4f1-4bdf-b4e7-50efdf8f0a85 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:31.241704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd2613-4d9a-46cd-ab69-8dfd9c9748ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:31.243791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd2613-4d9a-46cd-ab69-8dfd9c9748ac service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:31.259588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1202/2413] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:31.287350 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1a991be1-c4f1-4bdf-b4e7-50efdf8f0a85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '714f7c450ca945d4b2e149153ccfcdce'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.295579 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1a991be1-c4f1-4bdf-b4e7-50efdf8f0a85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '714f7c450ca945d4b2e149153ccfcdce'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.302596 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1a991be1-c4f1-4bdf-b4e7-50efdf8f0a85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '714f7c450ca945d4b2e149153ccfcdce'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.303243 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1a991be1-c4f1-4bdf-b4e7-50efdf8f0a85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '714f7c450ca945d4b2e149153ccfcdce'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.303867 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1a991be1-c4f1-4bdf-b4e7-50efdf8f0a85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '714f7c450ca945d4b2e149153ccfcdce'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.321716 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1212/2414] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:31 2022] DELETE /identity/v3/users/714f7c450ca945d4b2e149153ccfcdce => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:34:31.491986 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd2613-4d9a-46cd-ab69-8dfd9c9748ac service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:31.506276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd2613-4d9a-46cd-ab69-8dfd9c9748ac service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:31.573910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11b171-e8a5-4c8f-afcc-4c7e9c4ad610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:31.574607 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11b171-e8a5-4c8f-afcc-4c7e9c4ad610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:31.574900 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11b171-e8a5-4c8f-afcc-4c7e9c4ad610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:31.575339 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11b171-e8a5-4c8f-afcc-4c7e9c4ad610 admin admin] PATH_INFO: `/v3/projects/ef9eec76edd24a338dc9fdd966fceacd` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:31.576761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11b171-e8a5-4c8f-afcc-4c7e9c4ad610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef9eec76edd24a338dc9fdd966fceacd)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:31.579545 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11b171-e8a5-4c8f-afcc-4c7e9c4ad610 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:31.596819 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:31.656175 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ea11b171-e8a5-4c8f-afcc-4c7e9c4ad610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef9eec76edd24a338dc9fdd966fceacd', 'request_id': 'req-ea11b171-e8a5-4c8f-afcc-4c7e9c4ad610'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.663485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ea11b171-e8a5-4c8f-afcc-4c7e9c4ad610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef9eec76edd24a338dc9fdd966fceacd', 'request_id': 'req-ea11b171-e8a5-4c8f-afcc-4c7e9c4ad610'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.670717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ea11b171-e8a5-4c8f-afcc-4c7e9c4ad610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef9eec76edd24a338dc9fdd966fceacd', 'request_id': 'req-ea11b171-e8a5-4c8f-afcc-4c7e9c4ad610'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.682736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ea11b171-e8a5-4c8f-afcc-4c7e9c4ad610 admin admin] The token cache is being invalidate because project ef9eec76edd24a338dc9fdd966fceacd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:31.683025 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ea11b171-e8a5-4c8f-afcc-4c7e9c4ad610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.683686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ea11b171-e8a5-4c8f-afcc-4c7e9c4ad610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.685376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1203/2415] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:31 2022] DELETE /identity/v3/projects/ef9eec76edd24a338dc9fdd966fceacd => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:31.689153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a991be1-c4f1-4bdf-b4e7-50efdf8f0a85 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:31.691112 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1a991be1-c4f1-4bdf-b4e7-50efdf8f0a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:31.704599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a991be1-c4f1-4bdf-b4e7-50efdf8f0a85 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:31.762878 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54846771-23e8-4fe9-a437-7766cc911556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:31.763505 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54846771-23e8-4fe9-a437-7766cc911556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:31.763785 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54846771-23e8-4fe9-a437-7766cc911556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:31.764057 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54846771-23e8-4fe9-a437-7766cc911556 admin admin] PATH_INFO: `/v3/users/b9d9c312038e48dcb85f561e7feeb3b9` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:31.765173 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54846771-23e8-4fe9-a437-7766cc911556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9d9c312038e48dcb85f561e7feeb3b9)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:31.767131 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54846771-23e8-4fe9-a437-7766cc911556 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:31.769393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:31.813646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-54846771-23e8-4fe9-a437-7766cc911556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9d9c312038e48dcb85f561e7feeb3b9'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.821152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-54846771-23e8-4fe9-a437-7766cc911556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9d9c312038e48dcb85f561e7feeb3b9'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.827499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-54846771-23e8-4fe9-a437-7766cc911556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9d9c312038e48dcb85f561e7feeb3b9'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.828055 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-54846771-23e8-4fe9-a437-7766cc911556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9d9c312038e48dcb85f561e7feeb3b9'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.828679 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-54846771-23e8-4fe9-a437-7766cc911556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9d9c312038e48dcb85f561e7feeb3b9'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.845190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1213/2416] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:31 2022] DELETE /identity/v3/users/b9d9c312038e48dcb85f561e7feeb3b9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:32.032121 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11b171-e8a5-4c8f-afcc-4c7e9c4ad610 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:32.033648 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ea11b171-e8a5-4c8f-afcc-4c7e9c4ad610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:32.049638 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11b171-e8a5-4c8f-afcc-4c7e9c4ad610 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:32.117352 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea808b6-9686-47f9-b87b-7162abf4cdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:32.117977 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea808b6-9686-47f9-b87b-7162abf4cdce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:32.118273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea808b6-9686-47f9-b87b-7162abf4cdce admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:32.118559 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea808b6-9686-47f9-b87b-7162abf4cdce admin admin] PATH_INFO: `/v3/projects/ba732da5f3fe45ccb54f20b4be5d366f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:32.119679 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea808b6-9686-47f9-b87b-7162abf4cdce admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba732da5f3fe45ccb54f20b4be5d366f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:32.121605 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea808b6-9686-47f9-b87b-7162abf4cdce admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:32.168299 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eea808b6-9686-47f9-b87b-7162abf4cdce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba732da5f3fe45ccb54f20b4be5d366f', 'request_id': 'req-eea808b6-9686-47f9-b87b-7162abf4cdce'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:32.174085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eea808b6-9686-47f9-b87b-7162abf4cdce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba732da5f3fe45ccb54f20b4be5d366f', 'request_id': 'req-eea808b6-9686-47f9-b87b-7162abf4cdce'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:32.194314 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eea808b6-9686-47f9-b87b-7162abf4cdce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba732da5f3fe45ccb54f20b4be5d366f', 'request_id': 'req-eea808b6-9686-47f9-b87b-7162abf4cdce'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:32.195171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eea808b6-9686-47f9-b87b-7162abf4cdce admin admin] The token cache is being invalidate because project ba732da5f3fe45ccb54f20b4be5d366f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:32.195539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eea808b6-9686-47f9-b87b-7162abf4cdce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:32.196319 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eea808b6-9686-47f9-b87b-7162abf4cdce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:32.197813 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1204/2417] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:32 2022] DELETE /identity/v3/projects/ba732da5f3fe45ccb54f20b4be5d366f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:36.936396 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54846771-23e8-4fe9-a437-7766cc911556 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:36.937964 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-54846771-23e8-4fe9-a437-7766cc911556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:37.009180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54846771-23e8-4fe9-a437-7766cc911556 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:37.069059 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1a4ef-ed4a-430c-a5a1-7773c66831e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f62745b92a864480a9438de581e1d05e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:37.069583 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1a4ef-ed4a-430c-a5a1-7773c66831e3 service placement] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:37.069857 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1a4ef-ed4a-430c-a5a1-7773c66831e3 service placement] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:37.070164 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1a4ef-ed4a-430c-a5a1-7773c66831e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:37.071864 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-acb1a4ef-ed4a-430c-a5a1-7773c66831e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:37.082986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb1a4ef-ed4a-430c-a5a1-7773c66831e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:37.084571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb1a4ef-ed4a-430c-a5a1-7773c66831e3 service placement] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:37.094650 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1214/2418] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:40.311886 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87d801-765b-4f61-97bf-d858a8c8959c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:40.312394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87d801-765b-4f61-97bf-d858a8c8959c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:40.312499 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87d801-765b-4f61-97bf-d858a8c8959c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:40.331833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6e87d801-765b-4f61-97bf-d858a8c8959c None None] MFA Rules not processed for user `43dd72b161f8442285dea411741d66b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:40.333037 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6e87d801-765b-4f61-97bf-d858a8c8959c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:40.352860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:40.389201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1205/2419] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:34:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:47.090660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=83,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:50.382434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=13 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:32.161001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1a4ef-ed4a-430c-a5a1-7773c66831e3 service placement] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:32.162446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-acb1a4ef-ed4a-430c-a5a1-7773c66831e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:32.240314 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1a4ef-ed4a-430c-a5a1-7773c66831e3 service placement] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:32.243938 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299b45f-96ae-4c66-9cfa-99020bd42d44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:32.244532 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299b45f-96ae-4c66-9cfa-99020bd42d44 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:32.244810 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299b45f-96ae-4c66-9cfa-99020bd42d44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:32.245087 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299b45f-96ae-4c66-9cfa-99020bd42d44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:32.246808 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9299b45f-96ae-4c66-9cfa-99020bd42d44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:32.315578 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299b45f-96ae-4c66-9cfa-99020bd42d44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:32.316920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299b45f-96ae-4c66-9cfa-99020bd42d44 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:32.327684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1215/2420] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:41.531492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87d801-765b-4f61-97bf-d858a8c8959c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:41.532925 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6e87d801-765b-4f61-97bf-d858a8c8959c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:41.545543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87d801-765b-4f61-97bf-d858a8c8959c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:41.549079 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594964bc-2f53-4ae5-bb81-02b896d5d49e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:41.549672 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594964bc-2f53-4ae5-bb81-02b896d5d49e service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:41.549946 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594964bc-2f53-4ae5-bb81-02b896d5d49e service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:41.550270 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594964bc-2f53-4ae5-bb81-02b896d5d49e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:41.551958 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-594964bc-2f53-4ae5-bb81-02b896d5d49e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:41.565291 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594964bc-2f53-4ae5-bb81-02b896d5d49e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:41.566774 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594964bc-2f53-4ae5-bb81-02b896d5d49e service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:41.577578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1206/2421] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:42.322975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=63 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:51.572719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:03.788896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299b45f-96ae-4c66-9cfa-99020bd42d44 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:03.790299 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9299b45f-96ae-4c66-9cfa-99020bd42d44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:03.845309 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299b45f-96ae-4c66-9cfa-99020bd42d44 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:03.849442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cc86e-325c-4212-9217-d058c7aca030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:03.850037 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cc86e-325c-4212-9217-d058c7aca030 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:03.850345 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cc86e-325c-4212-9217-d058c7aca030 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:03.850623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cc86e-325c-4212-9217-d058c7aca030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:03.852321 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e54cc86e-325c-4212-9217-d058c7aca030 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:03.915439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cc86e-325c-4212-9217-d058c7aca030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:03.916889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cc86e-325c-4212-9217-d058c7aca030 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:03.927042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1216/2422] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:36:07.151267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594964bc-2f53-4ae5-bb81-02b896d5d49e service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:07.152728 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-594964bc-2f53-4ae5-bb81-02b896d5d49e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:07.167229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594964bc-2f53-4ae5-bb81-02b896d5d49e service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:07.171803 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54fc0f-6112-4204-b33f-bc7e2da5fa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:07.172420 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54fc0f-6112-4204-b33f-bc7e2da5fa6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:07.172700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54fc0f-6112-4204-b33f-bc7e2da5fa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:07.172974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54fc0f-6112-4204-b33f-bc7e2da5fa6d admin admin] PATH_INFO: `/v3/users/17286ff2e1764ec7b91e6274c7162437` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:07.186594 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54fc0f-6112-4204-b33f-bc7e2da5fa6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=17286ff2e1764ec7b91e6274c7162437)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:07.188545 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54fc0f-6112-4204-b33f-bc7e2da5fa6d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:07.232867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8f54fc0f-6112-4204-b33f-bc7e2da5fa6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17286ff2e1764ec7b91e6274c7162437'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:07.239650 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8f54fc0f-6112-4204-b33f-bc7e2da5fa6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17286ff2e1764ec7b91e6274c7162437'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:07.246594 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8f54fc0f-6112-4204-b33f-bc7e2da5fa6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17286ff2e1764ec7b91e6274c7162437'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:07.247214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8f54fc0f-6112-4204-b33f-bc7e2da5fa6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17286ff2e1764ec7b91e6274c7162437'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:07.247820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8f54fc0f-6112-4204-b33f-bc7e2da5fa6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17286ff2e1764ec7b91e6274c7162437'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:07.264610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1207/2423] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:36:07 2022] DELETE /identity/v3/users/17286ff2e1764ec7b91e6274c7162437 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:07.436981 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cc86e-325c-4212-9217-d058c7aca030 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:07.448287 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cc86e-325c-4212-9217-d058c7aca030 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:07.507841 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79cfdd-e038-4df9-ab35-56ec0b1a937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:07.508491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79cfdd-e038-4df9-ab35-56ec0b1a937c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:07.508764 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79cfdd-e038-4df9-ab35-56ec0b1a937c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:07.509029 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79cfdd-e038-4df9-ab35-56ec0b1a937c admin admin] PATH_INFO: `/v3/projects/9ae7098cc9f24d9dbc4695789c5b87b4` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:07.519389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79cfdd-e038-4df9-ab35-56ec0b1a937c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ae7098cc9f24d9dbc4695789c5b87b4)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:07.521881 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79cfdd-e038-4df9-ab35-56ec0b1a937c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:07.565338 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:07.574775 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4f79cfdd-e038-4df9-ab35-56ec0b1a937c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ae7098cc9f24d9dbc4695789c5b87b4', 'request_id': 'req-4f79cfdd-e038-4df9-ab35-56ec0b1a937c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:07.575248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4f79cfdd-e038-4df9-ab35-56ec0b1a937c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ae7098cc9f24d9dbc4695789c5b87b4', 'request_id': 'req-4f79cfdd-e038-4df9-ab35-56ec0b1a937c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:07.583293 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4f79cfdd-e038-4df9-ab35-56ec0b1a937c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ae7098cc9f24d9dbc4695789c5b87b4', 'request_id': 'req-4f79cfdd-e038-4df9-ab35-56ec0b1a937c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:07.595407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4f79cfdd-e038-4df9-ab35-56ec0b1a937c admin admin] The token cache is being invalidate because project 9ae7098cc9f24d9dbc4695789c5b87b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:36:07.595697 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4f79cfdd-e038-4df9-ab35-56ec0b1a937c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:07.596335 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4f79cfdd-e038-4df9-ab35-56ec0b1a937c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:07.598076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1217/2424] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:36:07 2022] DELETE /identity/v3/projects/9ae7098cc9f24d9dbc4695789c5b87b4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:07.602782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdca246-04ff-4062-a3bc-c39e54a6bb8b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:07.603248 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdca246-04ff-4062-a3bc-c39e54a6bb8b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:07.603522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdca246-04ff-4062-a3bc-c39e54a6bb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:07.622879 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7fdca246-04ff-4062-a3bc-c39e54a6bb8b None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:07.624072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7fdca246-04ff-4062-a3bc-c39e54a6bb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:07.684861 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1208/2425] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:07.694588 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e37618-bfdf-4685-a069-eadf3a7b71cd None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:07.694873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e37618-bfdf-4685-a069-eadf3a7b71cd None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:07.695152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e37618-bfdf-4685-a069-eadf3a7b71cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:07.714416 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-25e37618-bfdf-4685-a069-eadf3a7b71cd None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:07.715620 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-25e37618-bfdf-4685-a069-eadf3a7b71cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:07.723471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1218/2426] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:36:07.727594 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdca246-04ff-4062-a3bc-c39e54a6bb8b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:07.737335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdca246-04ff-4062-a3bc-c39e54a6bb8b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:07.742454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f0706-e1f5-459d-8ef8-f2b3ebceb359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:07.743044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f0706-e1f5-459d-8ef8-f2b3ebceb359 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:07.743325 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f0706-e1f5-459d-8ef8-f2b3ebceb359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:07.743596 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f0706-e1f5-459d-8ef8-f2b3ebceb359 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:07.744236 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f0706-e1f5-459d-8ef8-f2b3ebceb359 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:36:07.744539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f0706-e1f5-459d-8ef8-f2b3ebceb359 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:07.746131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f0706-e1f5-459d-8ef8-f2b3ebceb359 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:07.758159 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1209/2427] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:36:07.762150 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e37618-bfdf-4685-a069-eadf3a7b71cd None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:07.771658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e37618-bfdf-4685-a069-eadf3a7b71cd None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:07.776349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774da5f6-bbe1-45f7-9027-625c9831dc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:07.776939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774da5f6-bbe1-45f7-9027-625c9831dc42 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:07.777217 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774da5f6-bbe1-45f7-9027-625c9831dc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:07.777488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774da5f6-bbe1-45f7-9027-625c9831dc42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:07.778123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774da5f6-bbe1-45f7-9027-625c9831dc42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:07.780037 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774da5f6-bbe1-45f7-9027-625c9831dc42 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:07.821794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1219/2428] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:36:07 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:36:07.823094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f0706-e1f5-459d-8ef8-f2b3ebceb359 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:07.833272 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f0706-e1f5-459d-8ef8-f2b3ebceb359 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:07.881455 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:07.896958 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f99498-a573-49d6-96e3-5408e382c2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:07.897613 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f99498-a573-49d6-96e3-5408e382c2cf admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:07.897926 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f99498-a573-49d6-96e3-5408e382c2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:07.898192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f99498-a573-49d6-96e3-5408e382c2cf admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:07.900034 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f99498-a573-49d6-96e3-5408e382c2cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:07.903820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f99498-a573-49d6-96e3-5408e382c2cf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:07.932013 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1210/2429] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:36:07 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:36:07.933240 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774da5f6-bbe1-45f7-9027-625c9831dc42 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:07.944407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774da5f6-bbe1-45f7-9027-625c9831dc42 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:07.949014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b37d4-0b47-4254-bc43-3d3e459fc290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:07.949604 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b37d4-0b47-4254-bc43-3d3e459fc290 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:07.949882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b37d4-0b47-4254-bc43-3d3e459fc290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:07.950150 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b37d4-0b47-4254-bc43-3d3e459fc290 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:07.950821 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b37d4-0b47-4254-bc43-3d3e459fc290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:07.952395 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b37d4-0b47-4254-bc43-3d3e459fc290 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:07.962210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1220/2430] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:36:07.966118 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f99498-a573-49d6-96e3-5408e382c2cf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:07.976267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f99498-a573-49d6-96e3-5408e382c2cf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:07.980873 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec6fce-d50d-4686-9b13-67d872b3762f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:07.981454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec6fce-d50d-4686-9b13-67d872b3762f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:07.981728 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec6fce-d50d-4686-9b13-67d872b3762f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:07.981994 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec6fce-d50d-4686-9b13-67d872b3762f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:07.982658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec6fce-d50d-4686-9b13-67d872b3762f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:07.984240 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec6fce-d50d-4686-9b13-67d872b3762f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:07.995066 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1211/2431] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:36:07.999058 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b37d4-0b47-4254-bc43-3d3e459fc290 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:08.009605 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b37d4-0b47-4254-bc43-3d3e459fc290 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:08.014105 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda0acb-7039-42f6-82bd-d3922fd262fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:08.014766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda0acb-7039-42f6-82bd-d3922fd262fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:08.015046 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda0acb-7039-42f6-82bd-d3922fd262fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:08.015319 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda0acb-7039-42f6-82bd-d3922fd262fb admin admin] PATH_INFO: `/v3/projects/d29bcbc6c4aa4ad4b06de73033edca50/users/7173aaaacaca415b8fa32cd2524df672/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:08.031334 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda0acb-7039-42f6-82bd-d3922fd262fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d29bcbc6c4aa4ad4b06de73033edca50, user_id=7173aaaacaca415b8fa32cd2524df672, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:08.036390 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda0acb-7039-42f6-82bd-d3922fd262fb admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:08.048782 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1221/2432] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:36:07 2022] PUT /identity/v3/projects/d29bcbc6c4aa4ad4b06de73033edca50/users/7173aaaacaca415b8fa32cd2524df672/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:08.059788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa124d2-01ce-4bf7-a6d7-e3090ccd3d3a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:08.060103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa124d2-01ce-4bf7-a6d7-e3090ccd3d3a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:08.060380 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa124d2-01ce-4bf7-a6d7-e3090ccd3d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:08.087965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4aa124d2-01ce-4bf7-a6d7-e3090ccd3d3a None None] MFA Rules not processed for user `7173aaaacaca415b8fa32cd2524df672`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:08.089202 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4aa124d2-01ce-4bf7-a6d7-e3090ccd3d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:08.150882 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1212/2433] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:08.155273 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda0acb-7039-42f6-82bd-d3922fd262fb admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:08.215942 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda0acb-7039-42f6-82bd-d3922fd262fb admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:08.220621 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f594cf-023b-4e83-a233-85f540b0c84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:08.221212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f594cf-023b-4e83-a233-85f540b0c84a admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:08.221495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f594cf-023b-4e83-a233-85f540b0c84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:08.221788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f594cf-023b-4e83-a233-85f540b0c84a admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:08.222931 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f594cf-023b-4e83-a233-85f540b0c84a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:08.224812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f594cf-023b-4e83-a233-85f540b0c84a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:08.250908 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1222/2434] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:36:08 2022] POST /identity/v3/projects => generated 369 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:36:08.254966 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa124d2-01ce-4bf7-a6d7-e3090ccd3d3a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:08.265454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa124d2-01ce-4bf7-a6d7-e3090ccd3d3a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:08.321261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20e38f-8f21-40ca-9758-ce23fe254da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:08.321857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20e38f-8f21-40ca-9758-ce23fe254da1 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:08.322138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20e38f-8f21-40ca-9758-ce23fe254da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:08.322454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20e38f-8f21-40ca-9758-ce23fe254da1 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:08.323068 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20e38f-8f21-40ca-9758-ce23fe254da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:08.324982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20e38f-8f21-40ca-9758-ce23fe254da1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:08.345432 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1213/2435] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:36:08 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:36:08.349894 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f594cf-023b-4e83-a233-85f540b0c84a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:08.360136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f594cf-023b-4e83-a233-85f540b0c84a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:08.365243 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2f6f1-6e9d-4b6b-9cf5-eff5ef18a9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:08.365827 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2f6f1-6e9d-4b6b-9cf5-eff5ef18a9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:08.366105 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2f6f1-6e9d-4b6b-9cf5-eff5ef18a9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:08.366407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2f6f1-6e9d-4b6b-9cf5-eff5ef18a9e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:08.367090 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c2f6f1-6e9d-4b6b-9cf5-eff5ef18a9e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:08.368672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c2f6f1-6e9d-4b6b-9cf5-eff5ef18a9e7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:08.379489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1223/2436] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:36:08.383333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20e38f-8f21-40ca-9758-ce23fe254da1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:08.393134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20e38f-8f21-40ca-9758-ce23fe254da1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:08.398032 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d694b10-c96a-4723-bfb3-c267a7a24efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:08.398719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d694b10-c96a-4723-bfb3-c267a7a24efc admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:08.399073 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d694b10-c96a-4723-bfb3-c267a7a24efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:08.399349 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d694b10-c96a-4723-bfb3-c267a7a24efc admin admin] PATH_INFO: `/v3/projects/4566c5ece29942e3b683bb32cb86293e/users/1533aa5d1f38433fabb9d08e749cce84/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:08.414489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d694b10-c96a-4723-bfb3-c267a7a24efc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4566c5ece29942e3b683bb32cb86293e, user_id=1533aa5d1f38433fabb9d08e749cce84, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:08.417183 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d694b10-c96a-4723-bfb3-c267a7a24efc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:08.430600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1214/2437] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:36:08 2022] PUT /identity/v3/projects/4566c5ece29942e3b683bb32cb86293e/users/1533aa5d1f38433fabb9d08e749cce84/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:36:08.434537 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2f6f1-6e9d-4b6b-9cf5-eff5ef18a9e7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:08.444804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2f6f1-6e9d-4b6b-9cf5-eff5ef18a9e7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:08.474859 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:08.502160 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e337f7-b8e6-4202-a573-4f79187118ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:08.502785 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e337f7-b8e6-4202-a573-4f79187118ad admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:08.503061 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e337f7-b8e6-4202-a573-4f79187118ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:08.503329 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e337f7-b8e6-4202-a573-4f79187118ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:08.503957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e337f7-b8e6-4202-a573-4f79187118ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:08.505557 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e337f7-b8e6-4202-a573-4f79187118ad admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:08.515288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1224/2438] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:36:08.519923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d694b10-c96a-4723-bfb3-c267a7a24efc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:08.529930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d694b10-c96a-4723-bfb3-c267a7a24efc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:08.534558 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfb9e1-3a48-4bec-ae85-1d82655a98b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:08.535692 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfb9e1-3a48-4bec-ae85-1d82655a98b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:08.535940 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfb9e1-3a48-4bec-ae85-1d82655a98b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:08.536207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfb9e1-3a48-4bec-ae85-1d82655a98b3 admin admin] PATH_INFO: `/v3/projects/4566c5ece29942e3b683bb32cb86293e/users/1533aa5d1f38433fabb9d08e749cce84/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:08.538127 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfb9e1-3a48-4bec-ae85-1d82655a98b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4566c5ece29942e3b683bb32cb86293e, user_id=1533aa5d1f38433fabb9d08e749cce84, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:08.540782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfb9e1-3a48-4bec-ae85-1d82655a98b3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:08.551076 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1215/2439] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:36:08 2022] PUT /identity/v3/projects/4566c5ece29942e3b683bb32cb86293e/users/1533aa5d1f38433fabb9d08e749cce84/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:08.561375 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f9f20-1884-48bc-9805-bf9352c1a472 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:08.561670 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f9f20-1884-48bc-9805-bf9352c1a472 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:08.561947 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f9f20-1884-48bc-9805-bf9352c1a472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:08.589571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-138f9f20-1884-48bc-9805-bf9352c1a472 None None] MFA Rules not processed for user `1533aa5d1f38433fabb9d08e749cce84`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:08.590813 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-138f9f20-1884-48bc-9805-bf9352c1a472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:08.656644 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1225/2440] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:08.668261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfb9e1-3a48-4bec-ae85-1d82655a98b3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:08.669973 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cdbfb9e1-3a48-4bec-ae85-1d82655a98b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:08.680694 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:08.729236 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfb9e1-3a48-4bec-ae85-1d82655a98b3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:08.733012 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5cff6-6b8d-42ef-b9fa-2ba80047732a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:08.733608 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5cff6-6b8d-42ef-b9fa-2ba80047732a service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:08.734002 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5cff6-6b8d-42ef-b9fa-2ba80047732a service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:08.734304 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5cff6-6b8d-42ef-b9fa-2ba80047732a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:08.743558 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede5cff6-6b8d-42ef-b9fa-2ba80047732a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:08.744989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede5cff6-6b8d-42ef-b9fa-2ba80047732a service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:08.754545 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1216/2441] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:36:09.147100 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f9f20-1884-48bc-9805-bf9352c1a472 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:09.209635 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f9f20-1884-48bc-9805-bf9352c1a472 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:09.215068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e9073-f968-476d-92e1-7a239b70b9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:09.215688 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e9073-f968-476d-92e1-7a239b70b9b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:09.215967 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e9073-f968-476d-92e1-7a239b70b9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:09.216239 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e9073-f968-476d-92e1-7a239b70b9b1 admin admin] PATH_INFO: `/v3/users/7173aaaacaca415b8fa32cd2524df672` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:09.217356 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e9073-f968-476d-92e1-7a239b70b9b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7173aaaacaca415b8fa32cd2524df672)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:09.219316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e9073-f968-476d-92e1-7a239b70b9b1 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:09.264249 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7e9e9073-f968-476d-92e1-7a239b70b9b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7173aaaacaca415b8fa32cd2524df672'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:09.270976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7e9e9073-f968-476d-92e1-7a239b70b9b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7173aaaacaca415b8fa32cd2524df672'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:09.272566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:09.278556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7e9e9073-f968-476d-92e1-7a239b70b9b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7173aaaacaca415b8fa32cd2524df672'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:09.279273 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7e9e9073-f968-476d-92e1-7a239b70b9b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7173aaaacaca415b8fa32cd2524df672'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:09.280224 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7e9e9073-f968-476d-92e1-7a239b70b9b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7173aaaacaca415b8fa32cd2524df672'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:09.297713 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1226/2442] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:36:09 2022] DELETE /identity/v3/users/7173aaaacaca415b8fa32cd2524df672 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:09.302788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6c21a-49d3-4420-804f-8e1539d75208 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:09.303281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6c21a-49d3-4420-804f-8e1539d75208 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:09.303392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6c21a-49d3-4420-804f-8e1539d75208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:09.322726 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4cc6c21a-49d3-4420-804f-8e1539d75208 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:09.323924 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4cc6c21a-49d3-4420-804f-8e1539d75208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:09.380916 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1217/2443] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:09.391368 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e9073-f968-476d-92e1-7a239b70b9b1 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:09.393086 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7e9e9073-f968-476d-92e1-7a239b70b9b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:09.451060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e9073-f968-476d-92e1-7a239b70b9b1 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:09.454713 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb720b6d-ec69-4481-bb34-9a81841f8ddf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:09.455335 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb720b6d-ec69-4481-bb34-9a81841f8ddf service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:09.455614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb720b6d-ec69-4481-bb34-9a81841f8ddf service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:09.455887 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb720b6d-ec69-4481-bb34-9a81841f8ddf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:09.464765 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb720b6d-ec69-4481-bb34-9a81841f8ddf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:09.466171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb720b6d-ec69-4481-bb34-9a81841f8ddf service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:09.475469 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1227/2444] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:36:09.739462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6c21a-49d3-4420-804f-8e1539d75208 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:09.740906 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4cc6c21a-49d3-4420-804f-8e1539d75208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:09.752862 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6c21a-49d3-4420-804f-8e1539d75208 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:09.757415 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954e4d1-e36d-4b8e-9839-76660877ee59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:09.758149 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954e4d1-e36d-4b8e-9839-76660877ee59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:09.758471 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954e4d1-e36d-4b8e-9839-76660877ee59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:09.758744 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954e4d1-e36d-4b8e-9839-76660877ee59 admin admin] PATH_INFO: `/v3/projects/d29bcbc6c4aa4ad4b06de73033edca50` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:09.759998 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954e4d1-e36d-4b8e-9839-76660877ee59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d29bcbc6c4aa4ad4b06de73033edca50)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:09.761884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954e4d1-e36d-4b8e-9839-76660877ee59 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:09.808455 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6954e4d1-e36d-4b8e-9839-76660877ee59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd29bcbc6c4aa4ad4b06de73033edca50', 'request_id': 'req-6954e4d1-e36d-4b8e-9839-76660877ee59'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:09.813833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6954e4d1-e36d-4b8e-9839-76660877ee59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd29bcbc6c4aa4ad4b06de73033edca50', 'request_id': 'req-6954e4d1-e36d-4b8e-9839-76660877ee59'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:09.821282 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6954e4d1-e36d-4b8e-9839-76660877ee59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd29bcbc6c4aa4ad4b06de73033edca50', 'request_id': 'req-6954e4d1-e36d-4b8e-9839-76660877ee59'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:09.832465 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6954e4d1-e36d-4b8e-9839-76660877ee59 admin admin] The token cache is being invalidate because project d29bcbc6c4aa4ad4b06de73033edca50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:36:09.832755 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6954e4d1-e36d-4b8e-9839-76660877ee59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:09.833397 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6954e4d1-e36d-4b8e-9839-76660877ee59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:09.834917 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1218/2445] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:36:09 2022] DELETE /identity/v3/projects/d29bcbc6c4aa4ad4b06de73033edca50 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:09.838962 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb720b6d-ec69-4481-bb34-9a81841f8ddf service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:09.848951 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb720b6d-ec69-4481-bb34-9a81841f8ddf service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:09.903670 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6230590-ee88-46cc-8d0a-1859addba419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:09.904281 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6230590-ee88-46cc-8d0a-1859addba419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:09.904557 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6230590-ee88-46cc-8d0a-1859addba419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:09.904825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6230590-ee88-46cc-8d0a-1859addba419 admin admin] PATH_INFO: `/v3/users/1533aa5d1f38433fabb9d08e749cce84` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:09.905927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6230590-ee88-46cc-8d0a-1859addba419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1533aa5d1f38433fabb9d08e749cce84)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:09.907888 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6230590-ee88-46cc-8d0a-1859addba419 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:09.955123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b6230590-ee88-46cc-8d0a-1859addba419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1533aa5d1f38433fabb9d08e749cce84'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:09.962377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b6230590-ee88-46cc-8d0a-1859addba419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1533aa5d1f38433fabb9d08e749cce84'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:09.970217 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b6230590-ee88-46cc-8d0a-1859addba419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1533aa5d1f38433fabb9d08e749cce84'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:09.970896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b6230590-ee88-46cc-8d0a-1859addba419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1533aa5d1f38433fabb9d08e749cce84'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:09.971501 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b6230590-ee88-46cc-8d0a-1859addba419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1533aa5d1f38433fabb9d08e749cce84'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:09.983166 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:09.992033 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1228/2446] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:36:09 2022] DELETE /identity/v3/users/1533aa5d1f38433fabb9d08e749cce84 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:10.320569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954e4d1-e36d-4b8e-9839-76660877ee59 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:10.322098 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6954e4d1-e36d-4b8e-9839-76660877ee59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:10.336156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954e4d1-e36d-4b8e-9839-76660877ee59 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:10.381289 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:10.407402 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457c887-1a61-4202-9ce6-4ce15f830da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:10.408023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457c887-1a61-4202-9ce6-4ce15f830da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:10.408299 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457c887-1a61-4202-9ce6-4ce15f830da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:10.408567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457c887-1a61-4202-9ce6-4ce15f830da9 admin admin] PATH_INFO: `/v3/projects/4566c5ece29942e3b683bb32cb86293e` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:10.409672 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457c887-1a61-4202-9ce6-4ce15f830da9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4566c5ece29942e3b683bb32cb86293e)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:10.411633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457c887-1a61-4202-9ce6-4ce15f830da9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:10.459779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6457c887-1a61-4202-9ce6-4ce15f830da9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4566c5ece29942e3b683bb32cb86293e', 'request_id': 'req-6457c887-1a61-4202-9ce6-4ce15f830da9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:10.465192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6457c887-1a61-4202-9ce6-4ce15f830da9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4566c5ece29942e3b683bb32cb86293e', 'request_id': 'req-6457c887-1a61-4202-9ce6-4ce15f830da9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:10.472443 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6457c887-1a61-4202-9ce6-4ce15f830da9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4566c5ece29942e3b683bb32cb86293e', 'request_id': 'req-6457c887-1a61-4202-9ce6-4ce15f830da9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:10.484113 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6457c887-1a61-4202-9ce6-4ce15f830da9 admin admin] The token cache is being invalidate because project 4566c5ece29942e3b683bb32cb86293e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:36:10.484533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6457c887-1a61-4202-9ce6-4ce15f830da9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:10.485177 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6457c887-1a61-4202-9ce6-4ce15f830da9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:10.486746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1219/2447] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:36:10 2022] DELETE /identity/v3/projects/4566c5ece29942e3b683bb32cb86293e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:10.491684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22a802-73cf-4f01-a8a6-99d28a315360 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:10.492173 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22a802-73cf-4f01-a8a6-99d28a315360 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:10.492298 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22a802-73cf-4f01-a8a6-99d28a315360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:10.511969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6e22a802-73cf-4f01-a8a6-99d28a315360 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:10.513154 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6e22a802-73cf-4f01-a8a6-99d28a315360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:10.571088 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1229/2448] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:10.579751 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8378ba8-2543-4113-9d79-100a257ea102 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:10.580037 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8378ba8-2543-4113-9d79-100a257ea102 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:10.580314 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8378ba8-2543-4113-9d79-100a257ea102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:10.600493 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b8378ba8-2543-4113-9d79-100a257ea102 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:10.601719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b8378ba8-2543-4113-9d79-100a257ea102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:10.610047 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1220/2449] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:10.614508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22a802-73cf-4f01-a8a6-99d28a315360 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:10.625221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22a802-73cf-4f01-a8a6-99d28a315360 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:10.630927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95a56f-d10c-4d0e-a27e-62cc0b8e3c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:10.631516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95a56f-d10c-4d0e-a27e-62cc0b8e3c30 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:10.631795 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95a56f-d10c-4d0e-a27e-62cc0b8e3c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:10.632069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95a56f-d10c-4d0e-a27e-62cc0b8e3c30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:10.632718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95a56f-d10c-4d0e-a27e-62cc0b8e3c30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:36:10.633026 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95a56f-d10c-4d0e-a27e-62cc0b8e3c30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:10.634667 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95a56f-d10c-4d0e-a27e-62cc0b8e3c30 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:10.646699 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1230/2450] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:36:10.650820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8378ba8-2543-4113-9d79-100a257ea102 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:10.660274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8378ba8-2543-4113-9d79-100a257ea102 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:10.665756 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a8733-834e-4188-9803-79c9ca5e7513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:10.666413 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a8733-834e-4188-9803-79c9ca5e7513 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:10.666718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a8733-834e-4188-9803-79c9ca5e7513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:10.667144 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a8733-834e-4188-9803-79c9ca5e7513 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:10.667774 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a8733-834e-4188-9803-79c9ca5e7513 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:10.669908 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a8733-834e-4188-9803-79c9ca5e7513 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:10.697849 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1221/2451] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:36:10 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:36:10.701958 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95a56f-d10c-4d0e-a27e-62cc0b8e3c30 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:10.712276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95a56f-d10c-4d0e-a27e-62cc0b8e3c30 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:10.767717 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c17ad-e117-4291-a2b8-3c10e14c4087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:10.768304 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c17ad-e117-4291-a2b8-3c10e14c4087 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:10.768579 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c17ad-e117-4291-a2b8-3c10e14c4087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:10.768848 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c17ad-e117-4291-a2b8-3c10e14c4087 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:10.769466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c17ad-e117-4291-a2b8-3c10e14c4087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:10.771384 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c17ad-e117-4291-a2b8-3c10e14c4087 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:10.792413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1231/2452] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:36:10 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:36:10.796462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a8733-834e-4188-9803-79c9ca5e7513 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:10.808876 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a8733-834e-4188-9803-79c9ca5e7513 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:10.814136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bddc4e8-9966-40c9-91b4-a72825622063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:10.814749 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddc4e8-9966-40c9-91b4-a72825622063 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:10.815276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddc4e8-9966-40c9-91b4-a72825622063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:10.815539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddc4e8-9966-40c9-91b4-a72825622063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:10.816748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bddc4e8-9966-40c9-91b4-a72825622063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:10.818350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bddc4e8-9966-40c9-91b4-a72825622063 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:10.828048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1222/2453] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:36:10.832011 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c17ad-e117-4291-a2b8-3c10e14c4087 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:10.842004 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c17ad-e117-4291-a2b8-3c10e14c4087 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:10.846792 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bd2fa-600a-45a6-b298-40499fda330c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:10.847463 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bd2fa-600a-45a6-b298-40499fda330c admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:10.847802 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bd2fa-600a-45a6-b298-40499fda330c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:10.848133 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bd2fa-600a-45a6-b298-40499fda330c admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:10.848823 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bd2fa-600a-45a6-b298-40499fda330c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:10.850571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bd2fa-600a-45a6-b298-40499fda330c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:10.860688 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1232/2454] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:36:10.865046 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bddc4e8-9966-40c9-91b4-a72825622063 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:10.874931 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bddc4e8-9966-40c9-91b4-a72825622063 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:10.880517 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b7ecd-5b66-4dda-aa75-bf0c260c4461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:10.881204 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b7ecd-5b66-4dda-aa75-bf0c260c4461 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:10.881481 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b7ecd-5b66-4dda-aa75-bf0c260c4461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:10.881749 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b7ecd-5b66-4dda-aa75-bf0c260c4461 admin admin] PATH_INFO: `/v3/projects/8f2cac3102784da4b71bce01f0051d00/users/5515fcdbed514c8bb8fdb09bf07d9254/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:10.896094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b7ecd-5b66-4dda-aa75-bf0c260c4461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f2cac3102784da4b71bce01f0051d00, user_id=5515fcdbed514c8bb8fdb09bf07d9254, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:10.898797 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b7ecd-5b66-4dda-aa75-bf0c260c4461 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:10.910888 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1223/2455] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:36:10 2022] PUT /identity/v3/projects/8f2cac3102784da4b71bce01f0051d00/users/5515fcdbed514c8bb8fdb09bf07d9254/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:10.920921 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58857c-8adb-44c6-b88c-58bdd72143eb None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:10.921204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58857c-8adb-44c6-b88c-58bdd72143eb None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:10.921506 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58857c-8adb-44c6-b88c-58bdd72143eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:10.949099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-2b58857c-8adb-44c6-b88c-58bdd72143eb None None] MFA Rules not processed for user `5515fcdbed514c8bb8fdb09bf07d9254`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:10.950336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2b58857c-8adb-44c6-b88c-58bdd72143eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:10.997675 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:11.014746 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1233/2456] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:11.019673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b7ecd-5b66-4dda-aa75-bf0c260c4461 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:11.080091 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b7ecd-5b66-4dda-aa75-bf0c260c4461 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:11.085258 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce5561-358d-4a12-8fc4-8144e7704220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:11.085852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce5561-358d-4a12-8fc4-8144e7704220 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:11.086137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce5561-358d-4a12-8fc4-8144e7704220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:11.086436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce5561-358d-4a12-8fc4-8144e7704220 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:11.087059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce5561-358d-4a12-8fc4-8144e7704220 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:11.088954 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce5561-358d-4a12-8fc4-8144e7704220 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:11.094280 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:11.116117 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1224/2457] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:36:11 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:36:11.120228 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58857c-8adb-44c6-b88c-58bdd72143eb None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:11.131395 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58857c-8adb-44c6-b88c-58bdd72143eb None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:11.186189 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291657ad-91f1-48e8-8449-f2e8b5105ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:11.186817 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291657ad-91f1-48e8-8449-f2e8b5105ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:11.187097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291657ad-91f1-48e8-8449-f2e8b5105ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:11.187371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291657ad-91f1-48e8-8449-f2e8b5105ff0 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:11.187983 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291657ad-91f1-48e8-8449-f2e8b5105ff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:11.189897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291657ad-91f1-48e8-8449-f2e8b5105ff0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:11.211972 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1234/2458] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:36:11 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:36:11.216082 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce5561-358d-4a12-8fc4-8144e7704220 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:11.226780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce5561-358d-4a12-8fc4-8144e7704220 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:11.231399 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207f78f-a835-4170-b7f5-33c82e3c7648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:11.231985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207f78f-a835-4170-b7f5-33c82e3c7648 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:11.232264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207f78f-a835-4170-b7f5-33c82e3c7648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:11.232534 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207f78f-a835-4170-b7f5-33c82e3c7648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:11.233170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207f78f-a835-4170-b7f5-33c82e3c7648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:11.234794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207f78f-a835-4170-b7f5-33c82e3c7648 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:11.244507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1225/2459] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:36:11.248370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291657ad-91f1-48e8-8449-f2e8b5105ff0 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:11.258568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291657ad-91f1-48e8-8449-f2e8b5105ff0 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:11.263085 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725dc53-fbc8-4db6-94d4-92e266a99af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:11.263715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725dc53-fbc8-4db6-94d4-92e266a99af9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:11.263992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725dc53-fbc8-4db6-94d4-92e266a99af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:11.264262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725dc53-fbc8-4db6-94d4-92e266a99af9 admin admin] PATH_INFO: `/v3/projects/d120a21c3a3845989a2321c1dcb00596/users/6b5819bfc03d40b79c235f1cc89e1621/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:11.280123 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725dc53-fbc8-4db6-94d4-92e266a99af9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d120a21c3a3845989a2321c1dcb00596, user_id=6b5819bfc03d40b79c235f1cc89e1621, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:11.282824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725dc53-fbc8-4db6-94d4-92e266a99af9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:11.295052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1235/2460] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:36:11 2022] PUT /identity/v3/projects/d120a21c3a3845989a2321c1dcb00596/users/6b5819bfc03d40b79c235f1cc89e1621/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:11.298941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207f78f-a835-4170-b7f5-33c82e3c7648 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:11.309026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207f78f-a835-4170-b7f5-33c82e3c7648 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:11.364118 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf087c9-b238-46cf-83e4-9ad7164115f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:11.364770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf087c9-b238-46cf-83e4-9ad7164115f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:11.364965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf087c9-b238-46cf-83e4-9ad7164115f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:11.365231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf087c9-b238-46cf-83e4-9ad7164115f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:11.365855 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf087c9-b238-46cf-83e4-9ad7164115f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:11.367512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf087c9-b238-46cf-83e4-9ad7164115f5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:11.380767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1226/2461] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:36:11.382275 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725dc53-fbc8-4db6-94d4-92e266a99af9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:11.393155 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725dc53-fbc8-4db6-94d4-92e266a99af9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:11.398418 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0421a2-e8da-4cb6-8527-4e66cd53430a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:11.399058 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0421a2-e8da-4cb6-8527-4e66cd53430a admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:11.399352 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0421a2-e8da-4cb6-8527-4e66cd53430a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:11.399602 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0421a2-e8da-4cb6-8527-4e66cd53430a admin admin] PATH_INFO: `/v3/projects/d120a21c3a3845989a2321c1dcb00596/users/6b5819bfc03d40b79c235f1cc89e1621/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:11.401517 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0421a2-e8da-4cb6-8527-4e66cd53430a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d120a21c3a3845989a2321c1dcb00596, user_id=6b5819bfc03d40b79c235f1cc89e1621, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:11.404153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0421a2-e8da-4cb6-8527-4e66cd53430a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:11.414796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1236/2462] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:36:11 2022] PUT /identity/v3/projects/d120a21c3a3845989a2321c1dcb00596/users/6b5819bfc03d40b79c235f1cc89e1621/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:11.426547 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6906d-c380-47d8-b63a-c69a42477bdc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:11.426832 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6906d-c380-47d8-b63a-c69a42477bdc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:11.427103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6906d-c380-47d8-b63a-c69a42477bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:11.460671 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f6f6906d-c380-47d8-b63a-c69a42477bdc None None] MFA Rules not processed for user `6b5819bfc03d40b79c235f1cc89e1621`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:11.462489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f6f6906d-c380-47d8-b63a-c69a42477bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:11.543694 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1227/2463] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:11 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:11.555290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0421a2-e8da-4cb6-8527-4e66cd53430a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:11.556986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ad0421a2-e8da-4cb6-8527-4e66cd53430a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:11.614179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0421a2-e8da-4cb6-8527-4e66cd53430a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:11.617856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7054b0-0cec-4bec-a89c-739dc5f98c77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:11.618426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7054b0-0cec-4bec-a89c-739dc5f98c77 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:11.618703 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7054b0-0cec-4bec-a89c-739dc5f98c77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:11.618973 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7054b0-0cec-4bec-a89c-739dc5f98c77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:11.627785 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7054b0-0cec-4bec-a89c-739dc5f98c77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:11.629849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7054b0-0cec-4bec-a89c-739dc5f98c77 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:11.639195 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1237/2464] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:11 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:36:11.714872 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6906d-c380-47d8-b63a-c69a42477bdc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:11.724563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6906d-c380-47d8-b63a-c69a42477bdc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:11.728155 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877e59c-a05e-4513-8688-660a03ebd342 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:11.728740 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877e59c-a05e-4513-8688-660a03ebd342 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:11.729035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877e59c-a05e-4513-8688-660a03ebd342 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:11.729301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877e59c-a05e-4513-8688-660a03ebd342 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:11.782534 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:11.785453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e877e59c-a05e-4513-8688-660a03ebd342 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:11.787044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e877e59c-a05e-4513-8688-660a03ebd342 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:11.797609 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1228/2465] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:36:21.635004 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=74,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:21.794006 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:24.082220 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e31de-109f-4367-974b-a6a6b2db2733 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:24.082763 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e31de-109f-4367-974b-a6a6b2db2733 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:24.082856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e31de-109f-4367-974b-a6a6b2db2733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:24.087491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877e59c-a05e-4513-8688-660a03ebd342 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:24.089316 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e877e59c-a05e-4513-8688-660a03ebd342 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:24.151442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b61e31de-109f-4367-974b-a6a6b2db2733 None None] MFA Rules not processed for user `b88ccd3e5c4c48d3a27f4eaa7874e28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:24.152671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b61e31de-109f-4367-974b-a6a6b2db2733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:24.174447 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877e59c-a05e-4513-8688-660a03ebd342 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:24.180857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240c574-fed8-4735-a450-91f4fe409e46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:24.181467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240c574-fed8-4735-a450-91f4fe409e46 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:24.181746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240c574-fed8-4735-a450-91f4fe409e46 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:24.182022 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240c574-fed8-4735-a450-91f4fe409e46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:24.183796 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0240c574-fed8-4735-a450-91f4fe409e46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:24.197035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240c574-fed8-4735-a450-91f4fe409e46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:24.198720 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240c574-fed8-4735-a450-91f4fe409e46 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:24.209741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1229/2466] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:36:24.219255 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1238/2467] 10.222.0.39 () {58 vars in 1009 bytes} [Mon Aug 8 02:36:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:36:24.266517 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240c574-fed8-4735-a450-91f4fe409e46 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:24.267949 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0240c574-fed8-4735-a450-91f4fe409e46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:24.361739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240c574-fed8-4735-a450-91f4fe409e46 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:24.367694 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21033009-9aae-44ef-a9dd-db2617627e35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac60b4d621234225830c94edfac400e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:24.368464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21033009-9aae-44ef-a9dd-db2617627e35 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:24.368814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21033009-9aae-44ef-a9dd-db2617627e35 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:24.369159 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21033009-9aae-44ef-a9dd-db2617627e35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:24.377932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21033009-9aae-44ef-a9dd-db2617627e35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:24.380163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21033009-9aae-44ef-a9dd-db2617627e35 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:24.388375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1230/2468] 10.222.0.39 () {60 vars in 1384 bytes} [Mon Aug 8 02:36:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:36:26.343267 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e31de-109f-4367-974b-a6a6b2db2733 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:26.426623 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e31de-109f-4367-974b-a6a6b2db2733 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:26.431349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28c0d7-8a18-4538-94da-b03bd4e6f244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:26.432099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28c0d7-8a18-4538-94da-b03bd4e6f244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:26.432508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28c0d7-8a18-4538-94da-b03bd4e6f244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:26.432893 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28c0d7-8a18-4538-94da-b03bd4e6f244 admin admin] PATH_INFO: `/v3/users/5515fcdbed514c8bb8fdb09bf07d9254` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:26.434200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28c0d7-8a18-4538-94da-b03bd4e6f244 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5515fcdbed514c8bb8fdb09bf07d9254)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:26.436358 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28c0d7-8a18-4538-94da-b03bd4e6f244 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:26.493551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-6e28c0d7-8a18-4538-94da-b03bd4e6f244 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5515fcdbed514c8bb8fdb09bf07d9254'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:26.501574 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-6e28c0d7-8a18-4538-94da-b03bd4e6f244 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5515fcdbed514c8bb8fdb09bf07d9254'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:26.512152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-6e28c0d7-8a18-4538-94da-b03bd4e6f244 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5515fcdbed514c8bb8fdb09bf07d9254'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:26.513161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-6e28c0d7-8a18-4538-94da-b03bd4e6f244 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5515fcdbed514c8bb8fdb09bf07d9254'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:26.513989 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-6e28c0d7-8a18-4538-94da-b03bd4e6f244 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5515fcdbed514c8bb8fdb09bf07d9254'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:26.524050 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:26.544947 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1239/2469] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:36:26 2022] DELETE /identity/v3/users/5515fcdbed514c8bb8fdb09bf07d9254 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:26.549837 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157a075-2757-4f9d-8450-82940eeed957 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:26.550506 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157a075-2757-4f9d-8450-82940eeed957 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:26.550983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157a075-2757-4f9d-8450-82940eeed957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:26.575410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6157a075-2757-4f9d-8450-82940eeed957 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:26.576916 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6157a075-2757-4f9d-8450-82940eeed957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:26.631201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:26.644257 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1231/2470] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:26.653994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28c0d7-8a18-4538-94da-b03bd4e6f244 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:26.655540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6e28c0d7-8a18-4538-94da-b03bd4e6f244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:26.738017 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28c0d7-8a18-4538-94da-b03bd4e6f244 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:26.741970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf1fae-0b08-4a4e-ac26-38eed0f19862 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:26.742607 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf1fae-0b08-4a4e-ac26-38eed0f19862 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:26.742884 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf1fae-0b08-4a4e-ac26-38eed0f19862 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:26.743155 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf1fae-0b08-4a4e-ac26-38eed0f19862 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:26.756148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf1fae-0b08-4a4e-ac26-38eed0f19862 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:26.756148 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf1fae-0b08-4a4e-ac26-38eed0f19862 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:26.772462 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1240/2471] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:36:27.133615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157a075-2757-4f9d-8450-82940eeed957 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:27.135753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6157a075-2757-4f9d-8450-82940eeed957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:27.148525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157a075-2757-4f9d-8450-82940eeed957 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:27.154069 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c99c7-f623-4090-b8e1-2de5cd4f807e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:27.154773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c99c7-f623-4090-b8e1-2de5cd4f807e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:27.155050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c99c7-f623-4090-b8e1-2de5cd4f807e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:27.155317 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c99c7-f623-4090-b8e1-2de5cd4f807e admin admin] PATH_INFO: `/v3/projects/8f2cac3102784da4b71bce01f0051d00` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:27.156468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c99c7-f623-4090-b8e1-2de5cd4f807e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f2cac3102784da4b71bce01f0051d00)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:27.158431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c99c7-f623-4090-b8e1-2de5cd4f807e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:27.255567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c59c99c7-f623-4090-b8e1-2de5cd4f807e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f2cac3102784da4b71bce01f0051d00', 'request_id': 'req-c59c99c7-f623-4090-b8e1-2de5cd4f807e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.260985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c59c99c7-f623-4090-b8e1-2de5cd4f807e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f2cac3102784da4b71bce01f0051d00', 'request_id': 'req-c59c99c7-f623-4090-b8e1-2de5cd4f807e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.268223 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c59c99c7-f623-4090-b8e1-2de5cd4f807e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f2cac3102784da4b71bce01f0051d00', 'request_id': 'req-c59c99c7-f623-4090-b8e1-2de5cd4f807e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.282355 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c59c99c7-f623-4090-b8e1-2de5cd4f807e admin admin] The token cache is being invalidate because project 8f2cac3102784da4b71bce01f0051d00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:36:27.282681 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c59c99c7-f623-4090-b8e1-2de5cd4f807e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.283591 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c59c99c7-f623-4090-b8e1-2de5cd4f807e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.286567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1232/2472] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:36:27 2022] DELETE /identity/v3/projects/8f2cac3102784da4b71bce01f0051d00 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:27.290440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf1fae-0b08-4a4e-ac26-38eed0f19862 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:27.303904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf1fae-0b08-4a4e-ac26-38eed0f19862 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:27.370225 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39440531-4588-4a38-871b-92e94f76474e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:27.370936 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39440531-4588-4a38-871b-92e94f76474e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:27.371228 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39440531-4588-4a38-871b-92e94f76474e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:27.371447 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39440531-4588-4a38-871b-92e94f76474e admin admin] PATH_INFO: `/v3/users/6b5819bfc03d40b79c235f1cc89e1621` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:27.372577 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39440531-4588-4a38-871b-92e94f76474e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b5819bfc03d40b79c235f1cc89e1621)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:27.374520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39440531-4588-4a38-871b-92e94f76474e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:27.431583 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-39440531-4588-4a38-871b-92e94f76474e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b5819bfc03d40b79c235f1cc89e1621'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.441845 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-39440531-4588-4a38-871b-92e94f76474e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b5819bfc03d40b79c235f1cc89e1621'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.453200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-39440531-4588-4a38-871b-92e94f76474e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b5819bfc03d40b79c235f1cc89e1621'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.453837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-39440531-4588-4a38-871b-92e94f76474e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b5819bfc03d40b79c235f1cc89e1621'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.454488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-39440531-4588-4a38-871b-92e94f76474e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b5819bfc03d40b79c235f1cc89e1621'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.474264 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1241/2473] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:36:27 2022] DELETE /identity/v3/users/6b5819bfc03d40b79c235f1cc89e1621 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:27.818232 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c99c7-f623-4090-b8e1-2de5cd4f807e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:27.819829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c59c99c7-f623-4090-b8e1-2de5cd4f807e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:27.835523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c99c7-f623-4090-b8e1-2de5cd4f807e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:27.909323 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfe23f-af8f-4699-a7b9-d4febe6129f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:27.909979 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfe23f-af8f-4699-a7b9-d4febe6129f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:27.910345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfe23f-af8f-4699-a7b9-d4febe6129f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:27.910632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfe23f-af8f-4699-a7b9-d4febe6129f2 admin admin] PATH_INFO: `/v3/projects/d120a21c3a3845989a2321c1dcb00596` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:27.913247 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfe23f-af8f-4699-a7b9-d4febe6129f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d120a21c3a3845989a2321c1dcb00596)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:27.917081 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfe23f-af8f-4699-a7b9-d4febe6129f2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:27.997737 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-14cfe23f-af8f-4699-a7b9-d4febe6129f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd120a21c3a3845989a2321c1dcb00596', 'request_id': 'req-14cfe23f-af8f-4699-a7b9-d4febe6129f2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:28.010197 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-14cfe23f-af8f-4699-a7b9-d4febe6129f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd120a21c3a3845989a2321c1dcb00596', 'request_id': 'req-14cfe23f-af8f-4699-a7b9-d4febe6129f2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:28.021848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-14cfe23f-af8f-4699-a7b9-d4febe6129f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd120a21c3a3845989a2321c1dcb00596', 'request_id': 'req-14cfe23f-af8f-4699-a7b9-d4febe6129f2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:28.036624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-14cfe23f-af8f-4699-a7b9-d4febe6129f2 admin admin] The token cache is being invalidate because project d120a21c3a3845989a2321c1dcb00596 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:36:28.036909 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-14cfe23f-af8f-4699-a7b9-d4febe6129f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:28.037620 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-14cfe23f-af8f-4699-a7b9-d4febe6129f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:28.039304 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1233/2474] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:36:27 2022] DELETE /identity/v3/projects/d120a21c3a3845989a2321c1dcb00596 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:28.044271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9795a3b-f3ae-48fd-9a4f-3ade3df96939 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:28.044561 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9795a3b-f3ae-48fd-9a4f-3ade3df96939 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:28.044843 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9795a3b-f3ae-48fd-9a4f-3ade3df96939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:28.082771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c9795a3b-f3ae-48fd-9a4f-3ade3df96939 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:28.084355 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c9795a3b-f3ae-48fd-9a4f-3ade3df96939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:28.099482 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:28.168221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1242/2475] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:28.177467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6d386-1073-4fc7-a356-b880bc78af63 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:28.177743 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6d386-1073-4fc7-a356-b880bc78af63 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:28.178016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6d386-1073-4fc7-a356-b880bc78af63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:28.201848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-40f6d386-1073-4fc7-a356-b880bc78af63 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:28.203123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-40f6d386-1073-4fc7-a356-b880bc78af63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:28.215088 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1234/2476] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:28.219470 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9795a3b-f3ae-48fd-9a4f-3ade3df96939 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:28.230973 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9795a3b-f3ae-48fd-9a4f-3ade3df96939 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:28.236292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0867cee-dfaa-4d6d-9edf-5a1322b84a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:28.236901 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0867cee-dfaa-4d6d-9edf-5a1322b84a2a admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:28.237180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0867cee-dfaa-4d6d-9edf-5a1322b84a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:28.237973 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0867cee-dfaa-4d6d-9edf-5a1322b84a2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:28.238660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0867cee-dfaa-4d6d-9edf-5a1322b84a2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:36:28.238966 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0867cee-dfaa-4d6d-9edf-5a1322b84a2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:28.240644 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0867cee-dfaa-4d6d-9edf-5a1322b84a2a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:28.263914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1243/2477] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:36:28.265182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6d386-1073-4fc7-a356-b880bc78af63 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:28.270235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:28.276553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6d386-1073-4fc7-a356-b880bc78af63 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:28.282852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5292f6e-20a5-4ab4-9057-a8889cea556d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:28.283825 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5292f6e-20a5-4ab4-9057-a8889cea556d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:28.284958 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5292f6e-20a5-4ab4-9057-a8889cea556d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:28.285337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5292f6e-20a5-4ab4-9057-a8889cea556d admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:28.285980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5292f6e-20a5-4ab4-9057-a8889cea556d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:28.288653 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5292f6e-20a5-4ab4-9057-a8889cea556d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:28.324312 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1235/2478] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:36:28 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:36:28.326380 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0867cee-dfaa-4d6d-9edf-5a1322b84a2a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:28.336917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0867cee-dfaa-4d6d-9edf-5a1322b84a2a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:28.405882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7b022-cff2-44d3-808a-8d210270a814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:28.407850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7b022-cff2-44d3-808a-8d210270a814 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:28.407850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7b022-cff2-44d3-808a-8d210270a814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:28.407850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7b022-cff2-44d3-808a-8d210270a814 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:28.408540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7b022-cff2-44d3-808a-8d210270a814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:28.411099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7b022-cff2-44d3-808a-8d210270a814 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:28.438283 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1244/2479] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:36:28 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:36:28.441163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5292f6e-20a5-4ab4-9057-a8889cea556d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:28.454461 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5292f6e-20a5-4ab4-9057-a8889cea556d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:28.459584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a25ae-1c23-4a08-9d2c-36a91db89a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:28.460322 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a25ae-1c23-4a08-9d2c-36a91db89a4b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:28.460596 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a25ae-1c23-4a08-9d2c-36a91db89a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:28.460989 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a25ae-1c23-4a08-9d2c-36a91db89a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:28.461761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a25ae-1c23-4a08-9d2c-36a91db89a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:28.463455 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a25ae-1c23-4a08-9d2c-36a91db89a4b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:28.475813 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1236/2480] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:36:28.480203 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7b022-cff2-44d3-808a-8d210270a814 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:28.491684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7b022-cff2-44d3-808a-8d210270a814 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:28.497960 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8dd73d-76b5-4e7c-8c21-08686151c307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:28.498850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dd73d-76b5-4e7c-8c21-08686151c307 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:28.499274 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dd73d-76b5-4e7c-8c21-08686151c307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:28.499487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dd73d-76b5-4e7c-8c21-08686151c307 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:28.500105 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8dd73d-76b5-4e7c-8c21-08686151c307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:28.501850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8dd73d-76b5-4e7c-8c21-08686151c307 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:28.520171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1245/2481] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:36:28.524098 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a25ae-1c23-4a08-9d2c-36a91db89a4b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:28.535271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a25ae-1c23-4a08-9d2c-36a91db89a4b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:28.540263 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77c29b-7c61-4d38-9f18-0966e2584d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:28.540915 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77c29b-7c61-4d38-9f18-0966e2584d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:28.541199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77c29b-7c61-4d38-9f18-0966e2584d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:28.541470 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77c29b-7c61-4d38-9f18-0966e2584d14 admin admin] PATH_INFO: `/v3/projects/a27108cd23444bd7af44b85d3b2a5bc3/users/b2096d887aa448b5af4176cc3ac87dd3/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:28.563654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77c29b-7c61-4d38-9f18-0966e2584d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a27108cd23444bd7af44b85d3b2a5bc3, user_id=b2096d887aa448b5af4176cc3ac87dd3, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:28.566711 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77c29b-7c61-4d38-9f18-0966e2584d14 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:28.581555 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1237/2482] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:36:28 2022] PUT /identity/v3/projects/a27108cd23444bd7af44b85d3b2a5bc3/users/b2096d887aa448b5af4176cc3ac87dd3/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:28.591722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0cf3b-c58e-459e-a69d-c026a3717f0f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:28.592221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0cf3b-c58e-459e-a69d-c026a3717f0f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:28.592627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0cf3b-c58e-459e-a69d-c026a3717f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:28.625724 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-dbb0cf3b-c58e-459e-a69d-c026a3717f0f None None] MFA Rules not processed for user `b2096d887aa448b5af4176cc3ac87dd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:28.627140 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dbb0cf3b-c58e-459e-a69d-c026a3717f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:28.697925 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1246/2483] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:36:28.702536 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77c29b-7c61-4d38-9f18-0966e2584d14 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:28.770899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77c29b-7c61-4d38-9f18-0966e2584d14 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:28.775942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46220ca3-923e-462a-93ae-20cf800a829f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:28.776548 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46220ca3-923e-462a-93ae-20cf800a829f admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:28.776828 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46220ca3-923e-462a-93ae-20cf800a829f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:28.777101 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46220ca3-923e-462a-93ae-20cf800a829f admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:28.777713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46220ca3-923e-462a-93ae-20cf800a829f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:28.779659 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46220ca3-923e-462a-93ae-20cf800a829f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:28.813209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1238/2484] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:36:28 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:36:28.818047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0cf3b-c58e-459e-a69d-c026a3717f0f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:28.832341 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0cf3b-c58e-459e-a69d-c026a3717f0f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:28.837535 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:28.895865 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff581b-d5e9-4cfb-bbe8-0afa45fdde30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:28.896481 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff581b-d5e9-4cfb-bbe8-0afa45fdde30 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:28.896763 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff581b-d5e9-4cfb-bbe8-0afa45fdde30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:28.897041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff581b-d5e9-4cfb-bbe8-0afa45fdde30 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:28.897697 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff581b-d5e9-4cfb-bbe8-0afa45fdde30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:28.900115 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff581b-d5e9-4cfb-bbe8-0afa45fdde30 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:28.923215 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1247/2485] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:36:28 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:36:28.927750 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46220ca3-923e-462a-93ae-20cf800a829f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:28.938765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46220ca3-923e-462a-93ae-20cf800a829f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:28.949878 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccb5cf-283c-46bb-8bbb-5e334f7c944d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:28.950352 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccb5cf-283c-46bb-8bbb-5e334f7c944d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:28.950736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccb5cf-283c-46bb-8bbb-5e334f7c944d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:28.951111 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccb5cf-283c-46bb-8bbb-5e334f7c944d admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:28.951846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bccb5cf-283c-46bb-8bbb-5e334f7c944d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:28.953587 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bccb5cf-283c-46bb-8bbb-5e334f7c944d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:28.965840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1239/2486] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:36:28.970068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff581b-d5e9-4cfb-bbe8-0afa45fdde30 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:28.982000 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff581b-d5e9-4cfb-bbe8-0afa45fdde30 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:28.987061 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65833ea6-93b5-46b7-bd7f-3112acedb33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:28.987784 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65833ea6-93b5-46b7-bd7f-3112acedb33c admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:28.988147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65833ea6-93b5-46b7-bd7f-3112acedb33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:28.988513 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65833ea6-93b5-46b7-bd7f-3112acedb33c admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:28.989290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65833ea6-93b5-46b7-bd7f-3112acedb33c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:28.991021 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65833ea6-93b5-46b7-bd7f-3112acedb33c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:29.002898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1248/2487] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:36:29.007791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccb5cf-283c-46bb-8bbb-5e334f7c944d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:29.023633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccb5cf-283c-46bb-8bbb-5e334f7c944d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:29.029653 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f69713-366d-4bf4-b951-9acf08caa066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:29.030426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f69713-366d-4bf4-b951-9acf08caa066 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:29.030810 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f69713-366d-4bf4-b951-9acf08caa066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:29.031200 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f69713-366d-4bf4-b951-9acf08caa066 admin admin] PATH_INFO: `/v3/projects/00f7e93dc60f4393beb77258b0d0398a/users/8a9c35a15f3f4d7bb71d2c564a872c07/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:29.051480 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f69713-366d-4bf4-b951-9acf08caa066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00f7e93dc60f4393beb77258b0d0398a, user_id=8a9c35a15f3f4d7bb71d2c564a872c07, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:29.056107 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f69713-366d-4bf4-b951-9acf08caa066 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:29.079889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1240/2488] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:36:29 2022] PUT /identity/v3/projects/00f7e93dc60f4393beb77258b0d0398a/users/8a9c35a15f3f4d7bb71d2c564a872c07/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:29.099600 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68abd2-c42a-434f-b04e-e7d67d5ab6e3 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:29.100206 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68abd2-c42a-434f-b04e-e7d67d5ab6e3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:29.100840 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68abd2-c42a-434f-b04e-e7d67d5ab6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:29.141862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7a68abd2-c42a-434f-b04e-e7d67d5ab6e3 None None] MFA Rules not processed for user `8a9c35a15f3f4d7bb71d2c564a872c07`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:29.143724 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7a68abd2-c42a-434f-b04e-e7d67d5ab6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:29.245728 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1249/2489] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:29.249936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f69713-366d-4bf4-b951-9acf08caa066 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:29.317551 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:29.323149 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f69713-366d-4bf4-b951-9acf08caa066 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:29.330405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea7f0e-10c5-4cb7-897f-ebc4f762ab11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:29.331011 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea7f0e-10c5-4cb7-897f-ebc4f762ab11 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:29.331983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea7f0e-10c5-4cb7-897f-ebc4f762ab11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:29.332251 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea7f0e-10c5-4cb7-897f-ebc4f762ab11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:29.332880 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea7f0e-10c5-4cb7-897f-ebc4f762ab11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:29.334850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea7f0e-10c5-4cb7-897f-ebc4f762ab11 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:29.366290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1241/2490] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:36:29 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:36:29.370405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68abd2-c42a-434f-b04e-e7d67d5ab6e3 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:29.384232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68abd2-c42a-434f-b04e-e7d67d5ab6e3 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:29.464064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f058c04-08c3-4790-a964-30402a13074e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:29.464749 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f058c04-08c3-4790-a964-30402a13074e admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:29.464969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f058c04-08c3-4790-a964-30402a13074e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:29.465243 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f058c04-08c3-4790-a964-30402a13074e admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:29.465909 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f058c04-08c3-4790-a964-30402a13074e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:29.467926 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f058c04-08c3-4790-a964-30402a13074e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:29.490961 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1250/2491] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:36:29 2022] POST /identity/v3/users => generated 300 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:36:29.495223 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea7f0e-10c5-4cb7-897f-ebc4f762ab11 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:29.506226 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea7f0e-10c5-4cb7-897f-ebc4f762ab11 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:29.511834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe6d0d-2799-462d-beda-aa664aacca94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:29.512440 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe6d0d-2799-462d-beda-aa664aacca94 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:29.512748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe6d0d-2799-462d-beda-aa664aacca94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:29.512981 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe6d0d-2799-462d-beda-aa664aacca94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:29.513619 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbe6d0d-2799-462d-beda-aa664aacca94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:29.515299 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbe6d0d-2799-462d-beda-aa664aacca94 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:29.531870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1242/2492] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:36:29.537012 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f058c04-08c3-4790-a964-30402a13074e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:29.548304 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f058c04-08c3-4790-a964-30402a13074e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:29.553207 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c29218-9ce0-4c5b-9fd8-9503910a39e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:29.554003 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c29218-9ce0-4c5b-9fd8-9503910a39e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:29.554428 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c29218-9ce0-4c5b-9fd8-9503910a39e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:29.554806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c29218-9ce0-4c5b-9fd8-9503910a39e0 admin admin] PATH_INFO: `/v3/projects/a633f3ef88f64b898f91355b3365e89a/users/89e2445cfb484a64b22336e01b184b30/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:29.564246 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:29.583627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c29218-9ce0-4c5b-9fd8-9503910a39e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a633f3ef88f64b898f91355b3365e89a, user_id=89e2445cfb484a64b22336e01b184b30, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:29.586396 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c29218-9ce0-4c5b-9fd8-9503910a39e0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:29.602339 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1251/2493] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:36:29 2022] PUT /identity/v3/projects/a633f3ef88f64b898f91355b3365e89a/users/89e2445cfb484a64b22336e01b184b30/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:36:29.605579 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe6d0d-2799-462d-beda-aa664aacca94 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:29.616554 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe6d0d-2799-462d-beda-aa664aacca94 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:29.681593 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8260a2b-02ed-42f2-b6fa-951c9dcf3850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:29.682199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8260a2b-02ed-42f2-b6fa-951c9dcf3850 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:29.682507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8260a2b-02ed-42f2-b6fa-951c9dcf3850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:29.682779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8260a2b-02ed-42f2-b6fa-951c9dcf3850 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:29.683429 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8260a2b-02ed-42f2-b6fa-951c9dcf3850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:29.685104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8260a2b-02ed-42f2-b6fa-951c9dcf3850 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:29.696683 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1243/2494] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:36:29.700797 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c29218-9ce0-4c5b-9fd8-9503910a39e0 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:29.713632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c29218-9ce0-4c5b-9fd8-9503910a39e0 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:29.721344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e1e77-0d4a-47af-b59c-70fb5123348d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:29.722727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e1e77-0d4a-47af-b59c-70fb5123348d admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:29.723452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e1e77-0d4a-47af-b59c-70fb5123348d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:29.723452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e1e77-0d4a-47af-b59c-70fb5123348d admin admin] PATH_INFO: `/v3/projects/a633f3ef88f64b898f91355b3365e89a/users/89e2445cfb484a64b22336e01b184b30/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:29.725268 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e1e77-0d4a-47af-b59c-70fb5123348d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a633f3ef88f64b898f91355b3365e89a, user_id=89e2445cfb484a64b22336e01b184b30, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:29.728336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e1e77-0d4a-47af-b59c-70fb5123348d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:29.748935 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1252/2495] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:36:29 2022] PUT /identity/v3/projects/a633f3ef88f64b898f91355b3365e89a/users/89e2445cfb484a64b22336e01b184b30/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:36:29.751328 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684fa1c-7861-48bd-9357-79936a19db10 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:29.751808 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684fa1c-7861-48bd-9357-79936a19db10 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:29.752287 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684fa1c-7861-48bd-9357-79936a19db10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:29.788192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8684fa1c-7861-48bd-9357-79936a19db10 None None] MFA Rules not processed for user `89e2445cfb484a64b22336e01b184b30`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:29.790758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8684fa1c-7861-48bd-9357-79936a19db10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:29.877717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1244/2496] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:29.889877 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e1e77-0d4a-47af-b59c-70fb5123348d admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:29.892017 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ae6e1e77-0d4a-47af-b59c-70fb5123348d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:29.964829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e1e77-0d4a-47af-b59c-70fb5123348d admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:29.968927 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc712b-2e58-4782-8769-12b3c48fbe9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:29.969568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc712b-2e58-4782-8769-12b3c48fbe9f service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:29.969892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc712b-2e58-4782-8769-12b3c48fbe9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:29.970107 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc712b-2e58-4782-8769-12b3c48fbe9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:29.980774 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc712b-2e58-4782-8769-12b3c48fbe9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:29.982589 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc712b-2e58-4782-8769-12b3c48fbe9f service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:29.994051 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1253/2497] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:36:30.678233 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684fa1c-7861-48bd-9357-79936a19db10 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:30.679886 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8684fa1c-7861-48bd-9357-79936a19db10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:30.693964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684fa1c-7861-48bd-9357-79936a19db10 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:30.698446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88ab55-f3c7-46f6-b11b-343c1cc08d87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:30.699050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88ab55-f3c7-46f6-b11b-343c1cc08d87 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:30.699327 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88ab55-f3c7-46f6-b11b-343c1cc08d87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:30.699820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88ab55-f3c7-46f6-b11b-343c1cc08d87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:30.751438 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:30.765040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88ab55-f3c7-46f6-b11b-343c1cc08d87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:30.766544 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88ab55-f3c7-46f6-b11b-343c1cc08d87 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:30.776099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1245/2498] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:30 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:36:34.984098 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc712b-2e58-4782-8769-12b3c48fbe9f service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:34.992712 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc712b-2e58-4782-8769-12b3c48fbe9f service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:34.996375 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21757c-6a93-46f7-aecf-9b6b6570e8a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:34.996961 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21757c-6a93-46f7-aecf-9b6b6570e8a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:34.997237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21757c-6a93-46f7-aecf-9b6b6570e8a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:34.997504 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21757c-6a93-46f7-aecf-9b6b6570e8a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:35.054789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e21757c-6a93-46f7-aecf-9b6b6570e8a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:35.056158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e21757c-6a93-46f7-aecf-9b6b6570e8a0 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:35.064836 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1254/2499] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:36:35.138146 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88ab55-f3c7-46f6-b11b-343c1cc08d87 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:35.139925 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2c88ab55-f3c7-46f6-b11b-343c1cc08d87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:35.216511 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88ab55-f3c7-46f6-b11b-343c1cc08d87 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:35.220514 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9012e-c8f2-4acb-89af-2f0424791163 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:35.221122 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9012e-c8f2-4acb-89af-2f0424791163 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:35.221411 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9012e-c8f2-4acb-89af-2f0424791163 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:35.221669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9012e-c8f2-4acb-89af-2f0424791163 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:35.223641 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ccd9012e-c8f2-4acb-89af-2f0424791163 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:35.234392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9012e-c8f2-4acb-89af-2f0424791163 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:35.236161 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9012e-c8f2-4acb-89af-2f0424791163 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:35.245416 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1246/2500] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:36:37.196717 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21757c-6a93-46f7-aecf-9b6b6570e8a0 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:37.257048 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21757c-6a93-46f7-aecf-9b6b6570e8a0 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:37.261472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08121a1f-d4d9-4d91-86e8-3e895c56bfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:37.262083 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08121a1f-d4d9-4d91-86e8-3e895c56bfe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:37.262388 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08121a1f-d4d9-4d91-86e8-3e895c56bfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:37.262658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08121a1f-d4d9-4d91-86e8-3e895c56bfe7 admin admin] PATH_INFO: `/v3/users/b2096d887aa448b5af4176cc3ac87dd3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:37.263749 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08121a1f-d4d9-4d91-86e8-3e895c56bfe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2096d887aa448b5af4176cc3ac87dd3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:37.265635 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08121a1f-d4d9-4d91-86e8-3e895c56bfe7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:37.277365 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:37.311455 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-08121a1f-d4d9-4d91-86e8-3e895c56bfe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2096d887aa448b5af4176cc3ac87dd3'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:37.318813 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-08121a1f-d4d9-4d91-86e8-3e895c56bfe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2096d887aa448b5af4176cc3ac87dd3'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:37.326334 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-08121a1f-d4d9-4d91-86e8-3e895c56bfe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2096d887aa448b5af4176cc3ac87dd3'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:37.326952 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-08121a1f-d4d9-4d91-86e8-3e895c56bfe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2096d887aa448b5af4176cc3ac87dd3'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:37.327555 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-08121a1f-d4d9-4d91-86e8-3e895c56bfe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2096d887aa448b5af4176cc3ac87dd3'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:37.344678 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1255/2501] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:36:37 2022] DELETE /identity/v3/users/b2096d887aa448b5af4176cc3ac87dd3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:37.349628 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f505288-f4cf-4c2e-b362-dc59951b62a4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:37.350081 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f505288-f4cf-4c2e-b362-dc59951b62a4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:37.350345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f505288-f4cf-4c2e-b362-dc59951b62a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:37.369399 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3f505288-f4cf-4c2e-b362-dc59951b62a4 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:37.370627 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3f505288-f4cf-4c2e-b362-dc59951b62a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:37.428940 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1247/2502] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:37.438519 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08121a1f-d4d9-4d91-86e8-3e895c56bfe7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:37.439982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-08121a1f-d4d9-4d91-86e8-3e895c56bfe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:37.496606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08121a1f-d4d9-4d91-86e8-3e895c56bfe7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:37.500290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5d63b-a4d8-4fea-9d59-624281cf350d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:37.500899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5d63b-a4d8-4fea-9d59-624281cf350d service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:37.501167 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5d63b-a4d8-4fea-9d59-624281cf350d service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:37.501470 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5d63b-a4d8-4fea-9d59-624281cf350d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:37.510441 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5d63b-a4d8-4fea-9d59-624281cf350d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:37.511863 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5d63b-a4d8-4fea-9d59-624281cf350d service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:37.521166 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1256/2503] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:36:37.802623 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f505288-f4cf-4c2e-b362-dc59951b62a4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:37.808732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3f505288-f4cf-4c2e-b362-dc59951b62a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:37.823584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f505288-f4cf-4c2e-b362-dc59951b62a4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:37.829951 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cddbf7-02e4-48ab-9658-8614ac901ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:37.830587 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cddbf7-02e4-48ab-9658-8614ac901ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:37.830865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cddbf7-02e4-48ab-9658-8614ac901ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:37.831167 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cddbf7-02e4-48ab-9658-8614ac901ef9 admin admin] PATH_INFO: `/v3/projects/a27108cd23444bd7af44b85d3b2a5bc3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:37.832996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cddbf7-02e4-48ab-9658-8614ac901ef9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a27108cd23444bd7af44b85d3b2a5bc3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:37.834992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cddbf7-02e4-48ab-9658-8614ac901ef9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:37.881363 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-55cddbf7-02e4-48ab-9658-8614ac901ef9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a27108cd23444bd7af44b85d3b2a5bc3', 'request_id': 'req-55cddbf7-02e4-48ab-9658-8614ac901ef9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:37.886929 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-55cddbf7-02e4-48ab-9658-8614ac901ef9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a27108cd23444bd7af44b85d3b2a5bc3', 'request_id': 'req-55cddbf7-02e4-48ab-9658-8614ac901ef9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:37.893983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-55cddbf7-02e4-48ab-9658-8614ac901ef9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a27108cd23444bd7af44b85d3b2a5bc3', 'request_id': 'req-55cddbf7-02e4-48ab-9658-8614ac901ef9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:37.905479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-55cddbf7-02e4-48ab-9658-8614ac901ef9 admin admin] The token cache is being invalidate because project a27108cd23444bd7af44b85d3b2a5bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:36:37.905767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-55cddbf7-02e4-48ab-9658-8614ac901ef9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:37.906437 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-55cddbf7-02e4-48ab-9658-8614ac901ef9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:37.907916 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1248/2504] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:36:37 2022] DELETE /identity/v3/projects/a27108cd23444bd7af44b85d3b2a5bc3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:37.912826 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5d63b-a4d8-4fea-9d59-624281cf350d service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:37.922720 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5d63b-a4d8-4fea-9d59-624281cf350d service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:37.989132 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05427642-9ae3-4a1f-b553-34b3872dba02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:37.989771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05427642-9ae3-4a1f-b553-34b3872dba02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:37.990059 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05427642-9ae3-4a1f-b553-34b3872dba02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:37.994768 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05427642-9ae3-4a1f-b553-34b3872dba02 admin admin] PATH_INFO: `/v3/users/8a9c35a15f3f4d7bb71d2c564a872c07` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:37.996036 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05427642-9ae3-4a1f-b553-34b3872dba02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a9c35a15f3f4d7bb71d2c564a872c07)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:37.997961 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05427642-9ae3-4a1f-b553-34b3872dba02 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:38.058332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:38.069832 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-05427642-9ae3-4a1f-b553-34b3872dba02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a9c35a15f3f4d7bb71d2c564a872c07'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:38.083387 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-05427642-9ae3-4a1f-b553-34b3872dba02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a9c35a15f3f4d7bb71d2c564a872c07'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:38.103351 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-05427642-9ae3-4a1f-b553-34b3872dba02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a9c35a15f3f4d7bb71d2c564a872c07'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:38.104771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-05427642-9ae3-4a1f-b553-34b3872dba02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a9c35a15f3f4d7bb71d2c564a872c07'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:38.105759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-05427642-9ae3-4a1f-b553-34b3872dba02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a9c35a15f3f4d7bb71d2c564a872c07'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:38.133566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1257/2505] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:36:37 2022] DELETE /identity/v3/users/8a9c35a15f3f4d7bb71d2c564a872c07 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:38.408343 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cddbf7-02e4-48ab-9658-8614ac901ef9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:38.409980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-55cddbf7-02e4-48ab-9658-8614ac901ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:38.430998 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cddbf7-02e4-48ab-9658-8614ac901ef9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:38.450018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:38.499235 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4873833-5062-46fc-9fe1-97f4301e015b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:38.499782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4873833-5062-46fc-9fe1-97f4301e015b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:38.500069 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4873833-5062-46fc-9fe1-97f4301e015b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:38.500344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4873833-5062-46fc-9fe1-97f4301e015b admin admin] PATH_INFO: `/v3/projects/00f7e93dc60f4393beb77258b0d0398a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:38.501478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4873833-5062-46fc-9fe1-97f4301e015b admin admin] RBAC: Authorizing `identity:delete_project(project_id=00f7e93dc60f4393beb77258b0d0398a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:38.503444 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4873833-5062-46fc-9fe1-97f4301e015b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:38.554124 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4873833-5062-46fc-9fe1-97f4301e015b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00f7e93dc60f4393beb77258b0d0398a', 'request_id': 'req-c4873833-5062-46fc-9fe1-97f4301e015b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:38.560389 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4873833-5062-46fc-9fe1-97f4301e015b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00f7e93dc60f4393beb77258b0d0398a', 'request_id': 'req-c4873833-5062-46fc-9fe1-97f4301e015b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:38.569736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4873833-5062-46fc-9fe1-97f4301e015b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00f7e93dc60f4393beb77258b0d0398a', 'request_id': 'req-c4873833-5062-46fc-9fe1-97f4301e015b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:38.581352 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4873833-5062-46fc-9fe1-97f4301e015b admin admin] The token cache is being invalidate because project 00f7e93dc60f4393beb77258b0d0398a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:36:38.581627 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4873833-5062-46fc-9fe1-97f4301e015b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:38.582325 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4873833-5062-46fc-9fe1-97f4301e015b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:38.583911 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1249/2506] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:36:38 2022] DELETE /identity/v3/projects/00f7e93dc60f4393beb77258b0d0398a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:38.587851 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05427642-9ae3-4a1f-b553-34b3872dba02 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:38.589263 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-05427642-9ae3-4a1f-b553-34b3872dba02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:38.602031 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05427642-9ae3-4a1f-b553-34b3872dba02 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:38.660721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7e764-a1c0-4fa9-a0f9-d3cb1f8b0200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:38.661354 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7e764-a1c0-4fa9-a0f9-d3cb1f8b0200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:38.661636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7e764-a1c0-4fa9-a0f9-d3cb1f8b0200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:38.661910 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7e764-a1c0-4fa9-a0f9-d3cb1f8b0200 admin admin] PATH_INFO: `/v3/users/89e2445cfb484a64b22336e01b184b30` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:38.663081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7e764-a1c0-4fa9-a0f9-d3cb1f8b0200 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89e2445cfb484a64b22336e01b184b30)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:38.664994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7e764-a1c0-4fa9-a0f9-d3cb1f8b0200 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:38.712673 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3ac7e764-a1c0-4fa9-a0f9-d3cb1f8b0200 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89e2445cfb484a64b22336e01b184b30'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:38.719973 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3ac7e764-a1c0-4fa9-a0f9-d3cb1f8b0200 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89e2445cfb484a64b22336e01b184b30'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:38.727339 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3ac7e764-a1c0-4fa9-a0f9-d3cb1f8b0200 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89e2445cfb484a64b22336e01b184b30'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:38.728024 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3ac7e764-a1c0-4fa9-a0f9-d3cb1f8b0200 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89e2445cfb484a64b22336e01b184b30'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:38.728645 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3ac7e764-a1c0-4fa9-a0f9-d3cb1f8b0200 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89e2445cfb484a64b22336e01b184b30'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:38.745939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1258/2507] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:36:38 2022] DELETE /identity/v3/users/89e2445cfb484a64b22336e01b184b30 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:39.075798 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4873833-5062-46fc-9fe1-97f4301e015b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:39.077833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c4873833-5062-46fc-9fe1-97f4301e015b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:39.105998 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4873833-5062-46fc-9fe1-97f4301e015b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:39.168141 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e77518-17c3-46a5-ba53-d453d97c0449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:39.168770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e77518-17c3-46a5-ba53-d453d97c0449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:39.169053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e77518-17c3-46a5-ba53-d453d97c0449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:39.169330 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e77518-17c3-46a5-ba53-d453d97c0449 admin admin] PATH_INFO: `/v3/projects/a633f3ef88f64b898f91355b3365e89a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:39.170492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e77518-17c3-46a5-ba53-d453d97c0449 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a633f3ef88f64b898f91355b3365e89a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:39.172418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e77518-17c3-46a5-ba53-d453d97c0449 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:39.236274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:39.238054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f5e77518-17c3-46a5-ba53-d453d97c0449 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a633f3ef88f64b898f91355b3365e89a', 'request_id': 'req-f5e77518-17c3-46a5-ba53-d453d97c0449'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:39.245869 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f5e77518-17c3-46a5-ba53-d453d97c0449 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a633f3ef88f64b898f91355b3365e89a', 'request_id': 'req-f5e77518-17c3-46a5-ba53-d453d97c0449'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:39.255058 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f5e77518-17c3-46a5-ba53-d453d97c0449 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a633f3ef88f64b898f91355b3365e89a', 'request_id': 'req-f5e77518-17c3-46a5-ba53-d453d97c0449'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:39.266511 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f5e77518-17c3-46a5-ba53-d453d97c0449 admin admin] The token cache is being invalidate because project a633f3ef88f64b898f91355b3365e89a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:36:39.266732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f5e77518-17c3-46a5-ba53-d453d97c0449 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:39.267475 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f5e77518-17c3-46a5-ba53-d453d97c0449 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:39.269073 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1250/2508] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:36:39 2022] DELETE /identity/v3/projects/a633f3ef88f64b898f91355b3365e89a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:39.275633 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee259e34-5fee-4dc8-a9f2-60a60eefd889 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:39.275995 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee259e34-5fee-4dc8-a9f2-60a60eefd889 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:39.276394 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee259e34-5fee-4dc8-a9f2-60a60eefd889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:39.297281 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ee259e34-5fee-4dc8-a9f2-60a60eefd889 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:39.298627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ee259e34-5fee-4dc8-a9f2-60a60eefd889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:39.324788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:39.362479 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1259/2509] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:39.371895 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23a522-68eb-4e87-bde2-2814f188ad02 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:39.372178 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23a522-68eb-4e87-bde2-2814f188ad02 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:39.372447 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23a522-68eb-4e87-bde2-2814f188ad02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:39.391575 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6c23a522-68eb-4e87-bde2-2814f188ad02 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:39.392768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6c23a522-68eb-4e87-bde2-2814f188ad02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:39.401393 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1251/2510] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:39.406811 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee259e34-5fee-4dc8-a9f2-60a60eefd889 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:39.417655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee259e34-5fee-4dc8-a9f2-60a60eefd889 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:39.422216 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782829cf-fbb0-42bf-a35d-f313cd89f2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:39.422735 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782829cf-fbb0-42bf-a35d-f313cd89f2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:39.423024 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782829cf-fbb0-42bf-a35d-f313cd89f2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:39.423283 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782829cf-fbb0-42bf-a35d-f313cd89f2d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:39.423948 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782829cf-fbb0-42bf-a35d-f313cd89f2d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:36:39.424339 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782829cf-fbb0-42bf-a35d-f313cd89f2d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:39.425955 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782829cf-fbb0-42bf-a35d-f313cd89f2d6 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:39.437955 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1260/2511] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:36:39.443059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23a522-68eb-4e87-bde2-2814f188ad02 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:39.452523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23a522-68eb-4e87-bde2-2814f188ad02 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:39.457675 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5beb2-2b3c-4433-be75-d6c5c4ae6474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:39.458300 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5beb2-2b3c-4433-be75-d6c5c4ae6474 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:39.458582 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5beb2-2b3c-4433-be75-d6c5c4ae6474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:39.458881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5beb2-2b3c-4433-be75-d6c5c4ae6474 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:39.459512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf5beb2-2b3c-4433-be75-d6c5c4ae6474 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:39.461390 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf5beb2-2b3c-4433-be75-d6c5c4ae6474 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:39.489865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1252/2512] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:36:39 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:36:39.494001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782829cf-fbb0-42bf-a35d-f313cd89f2d6 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:39.504076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782829cf-fbb0-42bf-a35d-f313cd89f2d6 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:39.563147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76831b2a-8a1f-40cc-83a5-63c7c781be43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:39.563854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76831b2a-8a1f-40cc-83a5-63c7c781be43 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:39.564136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76831b2a-8a1f-40cc-83a5-63c7c781be43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:39.564407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76831b2a-8a1f-40cc-83a5-63c7c781be43 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:39.565291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76831b2a-8a1f-40cc-83a5-63c7c781be43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:39.567757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76831b2a-8a1f-40cc-83a5-63c7c781be43 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:39.593611 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1261/2513] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:36:39 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:36:39.598075 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5beb2-2b3c-4433-be75-d6c5c4ae6474 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:39.608345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5beb2-2b3c-4433-be75-d6c5c4ae6474 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:39.613055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580fa17-8356-4c18-8188-de4623ba0207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:39.613658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580fa17-8356-4c18-8188-de4623ba0207 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:39.613934 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580fa17-8356-4c18-8188-de4623ba0207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:39.614199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580fa17-8356-4c18-8188-de4623ba0207 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:39.614958 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b580fa17-8356-4c18-8188-de4623ba0207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:39.616470 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b580fa17-8356-4c18-8188-de4623ba0207 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:39.628328 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1253/2514] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:36:39.632332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76831b2a-8a1f-40cc-83a5-63c7c781be43 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:39.643428 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76831b2a-8a1f-40cc-83a5-63c7c781be43 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:39.649851 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf3683-8725-4492-9050-3baa4d399c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:39.651241 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf3683-8725-4492-9050-3baa4d399c58 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:39.651693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf3683-8725-4492-9050-3baa4d399c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:39.652013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf3683-8725-4492-9050-3baa4d399c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:39.652783 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf3683-8725-4492-9050-3baa4d399c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:39.655179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf3683-8725-4492-9050-3baa4d399c58 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:39.659458 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38277615-507e-48c0-8d4f-3a8ded0075d0 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:39.659790 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38277615-507e-48c0-8d4f-3a8ded0075d0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:39.660117 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38277615-507e-48c0-8d4f-3a8ded0075d0 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:39.661448 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1254/2515] 10.222.0.39 () {54 vars in 921 bytes} [Mon Aug 8 02:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:36:39.667308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb072808-3e53-4ceb-bed2-ceac03a8ea18 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:39.667585 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb072808-3e53-4ceb-bed2-ceac03a8ea18 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:39.667856 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb072808-3e53-4ceb-bed2-ceac03a8ea18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:39.676909 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1262/2516] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:36:39.678279 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf3683-8725-4492-9050-3baa4d399c58 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:39.688450 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf3683-8725-4492-9050-3baa4d399c58 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:39.693796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954a64b-50ae-4656-9426-34f4c8f78786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:39.694592 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954a64b-50ae-4656-9426-34f4c8f78786 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:39.695017 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954a64b-50ae-4656-9426-34f4c8f78786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:39.695296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954a64b-50ae-4656-9426-34f4c8f78786 admin admin] PATH_INFO: `/v3/projects/53c1b22f7ffe44dc939cab41ce14e321/users/2b1c75d77a80497597f99c0b5597ee27/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:39.708673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cb072808-3e53-4ceb-bed2-ceac03a8ea18 None None] MFA Rules not processed for user `c4238566a9c248ffbfa314f47f02670c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:39.709912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cb072808-3e53-4ceb-bed2-ceac03a8ea18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:39.713302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954a64b-50ae-4656-9426-34f4c8f78786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53c1b22f7ffe44dc939cab41ce14e321, user_id=2b1c75d77a80497597f99c0b5597ee27, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:39.716055 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954a64b-50ae-4656-9426-34f4c8f78786 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:39.731191 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1263/2517] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:36:39 2022] PUT /identity/v3/projects/53c1b22f7ffe44dc939cab41ce14e321/users/2b1c75d77a80497597f99c0b5597ee27/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:36:39.742557 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641110a-a64f-43ce-b8a2-11a70d108df7 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:39.742867 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641110a-a64f-43ce-b8a2-11a70d108df7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:39.743108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641110a-a64f-43ce-b8a2-11a70d108df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:39.786354 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1255/2518] 10.222.0.39 () {58 vars in 1018 bytes} [Mon Aug 8 02:36:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:36:39.799779 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb072808-3e53-4ceb-bed2-ceac03a8ea18 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:39.801277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cb072808-3e53-4ceb-bed2-ceac03a8ea18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:39.862485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb072808-3e53-4ceb-bed2-ceac03a8ea18 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:39.893017 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:39.907151 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-9641110a-a64f-43ce-b8a2-11a70d108df7 None None] MFA Rules not processed for user `2b1c75d77a80497597f99c0b5597ee27`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:39.908366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9641110a-a64f-43ce-b8a2-11a70d108df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:39.917738 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31218c98-6b54-4ed5-9263-d4579f50ddc6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7776a6a1faf34b369e399d83bb3e18c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:39.918812 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31218c98-6b54-4ed5-9263-d4579f50ddc6 service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:39.919331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31218c98-6b54-4ed5-9263-d4579f50ddc6 service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:39.919827 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31218c98-6b54-4ed5-9263-d4579f50ddc6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:39.973884 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1264/2519] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:36:39.978414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641110a-a64f-43ce-b8a2-11a70d108df7 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:39.982113 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:39.991474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31218c98-6b54-4ed5-9263-d4579f50ddc6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:39.992913 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31218c98-6b54-4ed5-9263-d4579f50ddc6 service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:40.004477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1256/2520] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:36:40.042973 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641110a-a64f-43ce-b8a2-11a70d108df7 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:40.047393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d1af9-20ae-44d0-a22c-48c68efc072b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:40.047989 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d1af9-20ae-44d0-a22c-48c68efc072b admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:40.048259 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d1af9-20ae-44d0-a22c-48c68efc072b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:40.048523 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d1af9-20ae-44d0-a22c-48c68efc072b admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:40.049141 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d1af9-20ae-44d0-a22c-48c68efc072b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:40.051547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d1af9-20ae-44d0-a22c-48c68efc072b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:40.077539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde2a38-83ea-44b0-a6ba-1ef8fa3565cd None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:40.077924 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde2a38-83ea-44b0-a6ba-1ef8fa3565cd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:40.078205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde2a38-83ea-44b0-a6ba-1ef8fa3565cd None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:40.079596 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1257/2521] 10.222.0.39 () {54 vars in 921 bytes} [Mon Aug 8 02:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:36:40.083576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1265/2522] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:36:39 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:36:40.084400 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0ee43-1622-42af-855d-26155f0bffb6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:40.084660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d1af9-20ae-44d0-a22c-48c68efc072b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:40.084787 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0ee43-1622-42af-855d-26155f0bffb6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:40.084934 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0ee43-1622-42af-855d-26155f0bffb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:40.123430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b1a0ee43-1622-42af-855d-26155f0bffb6 None None] MFA Rules not processed for user `78506fea1c024b169a4b7ee86116658c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:40.124646 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b1a0ee43-1622-42af-855d-26155f0bffb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:40.151807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d1af9-20ae-44d0-a22c-48c68efc072b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:40.158487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58db8f-3507-42a3-9b28-67470f204036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:40.159080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58db8f-3507-42a3-9b28-67470f204036 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:40.159354 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58db8f-3507-42a3-9b28-67470f204036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:40.159622 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58db8f-3507-42a3-9b28-67470f204036 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:40.160261 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58db8f-3507-42a3-9b28-67470f204036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:40.162814 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58db8f-3507-42a3-9b28-67470f204036 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:40.178112 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1258/2523] 10.222.0.39 () {58 vars in 1018 bytes} [Mon Aug 8 02:36:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:36:40.192325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1266/2524] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:36:40 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:36:40.192502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0ee43-1622-42af-855d-26155f0bffb6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:40.193332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58db8f-3507-42a3-9b28-67470f204036 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:40.193551 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b1a0ee43-1622-42af-855d-26155f0bffb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:40.202895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58db8f-3507-42a3-9b28-67470f204036 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:40.203760 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0ee43-1622-42af-855d-26155f0bffb6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:40.207558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb1470-92f2-4d8c-b87a-b0ee7ea9289e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:40.207822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04f353-0282-4f7c-909a-c38fd38fba7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:40.208133 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb1470-92f2-4d8c-b87a-b0ee7ea9289e admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:40.208406 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04f353-0282-4f7c-909a-c38fd38fba7e service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:40.208513 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb1470-92f2-4d8c-b87a-b0ee7ea9289e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:40.208668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb1470-92f2-4d8c-b87a-b0ee7ea9289e admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:40.208773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04f353-0282-4f7c-909a-c38fd38fba7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:40.208933 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04f353-0282-4f7c-909a-c38fd38fba7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:40.209297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb1470-92f2-4d8c-b87a-b0ee7ea9289e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:40.210898 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb1470-92f2-4d8c-b87a-b0ee7ea9289e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:40.216895 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04f353-0282-4f7c-909a-c38fd38fba7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:40.218417 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04f353-0282-4f7c-909a-c38fd38fba7e service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:40.221707 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1267/2525] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:36:40.226000 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb1470-92f2-4d8c-b87a-b0ee7ea9289e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:40.227399 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1259/2526] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:36:40.236422 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb1470-92f2-4d8c-b87a-b0ee7ea9289e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:40.241780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b45fd7-093c-459c-bf26-9d2ff713d11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:40.242530 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b45fd7-093c-459c-bf26-9d2ff713d11a admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:40.242874 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b45fd7-093c-459c-bf26-9d2ff713d11a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:40.243210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b45fd7-093c-459c-bf26-9d2ff713d11a admin admin] PATH_INFO: `/v3/projects/11673bc0bf7e4a4c86151b5bddfd1e8f/users/6fde2c155d504055918e4a46a1fc1503/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:40.258292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b45fd7-093c-459c-bf26-9d2ff713d11a admin admin] RBAC: Authorizing `identity:create_grant(project_id=11673bc0bf7e4a4c86151b5bddfd1e8f, user_id=6fde2c155d504055918e4a46a1fc1503, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:40.261002 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b45fd7-093c-459c-bf26-9d2ff713d11a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:40.274438 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1268/2527] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:36:40 2022] PUT /identity/v3/projects/11673bc0bf7e4a4c86151b5bddfd1e8f/users/6fde2c155d504055918e4a46a1fc1503/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:40.278352 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04f353-0282-4f7c-909a-c38fd38fba7e service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:40.290046 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04f353-0282-4f7c-909a-c38fd38fba7e service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:40.314449 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2a95c-c44f-4f74-b04c-15a3e09b0cd7 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:40.314739 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2a95c-c44f-4f74-b04c-15a3e09b0cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:40.315257 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2a95c-c44f-4f74-b04c-15a3e09b0cd7 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:40.316635 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1269/2528] 10.222.0.39 () {54 vars in 921 bytes} [Mon Aug 8 02:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:36:40.323050 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cbbe3-fb8d-4648-a69d-1e0d04ef5f19 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:40.323399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cbbe3-fb8d-4648-a69d-1e0d04ef5f19 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:40.323681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cbbe3-fb8d-4648-a69d-1e0d04ef5f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:40.343362 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-3e6cbbe3-fb8d-4648-a69d-1e0d04ef5f19 None None] MFA Rules not processed for user `c4238566a9c248ffbfa314f47f02670c`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:40.344103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:40.344579 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3e6cbbe3-fb8d-4648-a69d-1e0d04ef5f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:40.352496 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e28cf-f829-4e6b-ab1b-6071546fc1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:40.353098 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e28cf-f829-4e6b-ab1b-6071546fc1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:40.353375 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e28cf-f829-4e6b-ab1b-6071546fc1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:40.353647 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e28cf-f829-4e6b-ab1b-6071546fc1a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:40.354307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e28cf-f829-4e6b-ab1b-6071546fc1a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:40.355926 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e28cf-f829-4e6b-ab1b-6071546fc1a2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:40.366359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1260/2529] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:36:40.370341 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e28cf-f829-4e6b-ab1b-6071546fc1a2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:40.380225 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e28cf-f829-4e6b-ab1b-6071546fc1a2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:40.384843 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32fdca-39f5-410d-b3c6-4c9d0a6fcc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:40.385480 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32fdca-39f5-410d-b3c6-4c9d0a6fcc40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:40.385761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32fdca-39f5-410d-b3c6-4c9d0a6fcc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:40.386032 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32fdca-39f5-410d-b3c6-4c9d0a6fcc40 admin admin] PATH_INFO: `/v3/projects/11673bc0bf7e4a4c86151b5bddfd1e8f/users/6fde2c155d504055918e4a46a1fc1503/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:40.387073 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:40.387990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32fdca-39f5-410d-b3c6-4c9d0a6fcc40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11673bc0bf7e4a4c86151b5bddfd1e8f, user_id=6fde2c155d504055918e4a46a1fc1503, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:40.391103 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32fdca-39f5-410d-b3c6-4c9d0a6fcc40 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:40.402751 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1261/2530] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:36:40 2022] PUT /identity/v3/projects/11673bc0bf7e4a4c86151b5bddfd1e8f/users/6fde2c155d504055918e4a46a1fc1503/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:36:40.413702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80000dd3-d4c4-4ea9-816e-062a929ba984 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:40.414018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80000dd3-d4c4-4ea9-816e-062a929ba984 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:40.414342 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80000dd3-d4c4-4ea9-816e-062a929ba984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:40.418496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1270/2531] 10.222.0.39 () {58 vars in 1018 bytes} [Mon Aug 8 02:36:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:40.435212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6cbbe3-fb8d-4648-a69d-1e0d04ef5f19 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:40.443620 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-80000dd3-d4c4-4ea9-816e-062a929ba984 None None] MFA Rules not processed for user `6fde2c155d504055918e4a46a1fc1503`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:40.444833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-80000dd3-d4c4-4ea9-816e-062a929ba984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:40.494492 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6cbbe3-fb8d-4648-a69d-1e0d04ef5f19 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:40.515417 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1262/2532] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:40 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:36:40.527184 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80000dd3-d4c4-4ea9-816e-062a929ba984 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:40.528654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-80000dd3-d4c4-4ea9-816e-062a929ba984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:40.543084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3a133-6272-4eda-8b43-987148063151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:40.543700 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3a133-6272-4eda-8b43-987148063151 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:40.543973 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3a133-6272-4eda-8b43-987148063151 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:40.544237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3a133-6272-4eda-8b43-987148063151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:40.591408 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80000dd3-d4c4-4ea9-816e-062a929ba984 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:40.597893 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37035f8f-3a3a-4d6d-b2b9-413fb68f2d60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:40.598386 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37035f8f-3a3a-4d6d-b2b9-413fb68f2d60 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:40.598777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37035f8f-3a3a-4d6d-b2b9-413fb68f2d60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:40.599230 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37035f8f-3a3a-4d6d-b2b9-413fb68f2d60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:40.611752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37035f8f-3a3a-4d6d-b2b9-413fb68f2d60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:40.611752 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37035f8f-3a3a-4d6d-b2b9-413fb68f2d60 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:40.618789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3a133-6272-4eda-8b43-987148063151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:40.620275 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3a133-6272-4eda-8b43-987148063151 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:40.630562 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1263/2533] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:40 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:36:40.632836 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1271/2534] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:36:40.703666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37035f8f-3a3a-4d6d-b2b9-413fb68f2d60 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:40.706278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-37035f8f-3a3a-4d6d-b2b9-413fb68f2d60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:40.725219 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3a133-6272-4eda-8b43-987148063151 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:40.735670 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3a133-6272-4eda-8b43-987148063151 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:40.740171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08977ef-af37-42ba-b29d-5bac5e6b27d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:40.740767 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08977ef-af37-42ba-b29d-5bac5e6b27d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:40.741048 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08977ef-af37-42ba-b29d-5bac5e6b27d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:40.741316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08977ef-af37-42ba-b29d-5bac5e6b27d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:40.778562 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:40.787809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37035f8f-3a3a-4d6d-b2b9-413fb68f2d60 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:40.793621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4094c6a-ca61-4683-a3bc-9873fb700a7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f62745b92a864480a9438de581e1d05e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:40.794156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4094c6a-ca61-4683-a3bc-9873fb700a7f service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:40.794532 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4094c6a-ca61-4683-a3bc-9873fb700a7f service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:40.794800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4094c6a-ca61-4683-a3bc-9873fb700a7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:40.796949 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d4094c6a-ca61-4683-a3bc-9873fb700a7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:40.808369 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:40.811185 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08977ef-af37-42ba-b29d-5bac5e6b27d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:40.811944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4094c6a-ca61-4683-a3bc-9873fb700a7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:40.812754 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08977ef-af37-42ba-b29d-5bac5e6b27d3 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:40.814808 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4094c6a-ca61-4683-a3bc-9873fb700a7f service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:40.825322 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1272/2535] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:36:40.832223 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1264/2536] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:36:44.314551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4702d-88fe-4e6e-8f21-b8cb8409a8a3 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:44.315272 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4702d-88fe-4e6e-8f21-b8cb8409a8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:44.315383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4702d-88fe-4e6e-8f21-b8cb8409a8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:44.333931 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-aeb4702d-88fe-4e6e-8f21-b8cb8409a8a3 None None] MFA Rules not processed for user `b88ccd3e5c4c48d3a27f4eaa7874e28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:44.335143 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-aeb4702d-88fe-4e6e-8f21-b8cb8409a8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:44.389362 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1273/2537] 10.222.0.39 () {58 vars in 1009 bytes} [Mon Aug 8 02:36:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:44.401519 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4094c6a-ca61-4683-a3bc-9873fb700a7f service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:44.402966 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d4094c6a-ca61-4683-a3bc-9873fb700a7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:44.471882 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4094c6a-ca61-4683-a3bc-9873fb700a7f service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:44.477496 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ea319-d366-4fb8-bc03-e7a376042601 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac60b4d621234225830c94edfac400e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:44.478092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ea319-d366-4fb8-bc03-e7a376042601 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:44.478399 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ea319-d366-4fb8-bc03-e7a376042601 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:44.478668 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ea319-d366-4fb8-bc03-e7a376042601 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:44.486985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ea319-d366-4fb8-bc03-e7a376042601 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:44.488574 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ea319-d366-4fb8-bc03-e7a376042601 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:44.496763 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1265/2538] 10.222.0.39 () {60 vars in 1384 bytes} [Mon Aug 8 02:36:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:36:47.025776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4702d-88fe-4e6e-8f21-b8cb8409a8a3 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:47.127410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4702d-88fe-4e6e-8f21-b8cb8409a8a3 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:47.132247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8f165-4d57-4d55-b713-237366f007c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:47.132960 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8f165-4d57-4d55-b713-237366f007c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:47.133343 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8f165-4d57-4d55-b713-237366f007c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:47.133717 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8f165-4d57-4d55-b713-237366f007c1 admin admin] PATH_INFO: `/v3/users/2b1c75d77a80497597f99c0b5597ee27` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:47.134997 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e8f165-4d57-4d55-b713-237366f007c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b1c75d77a80497597f99c0b5597ee27)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:47.137023 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e8f165-4d57-4d55-b713-237366f007c1 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:47.192989 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a6e8f165-4d57-4d55-b713-237366f007c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b1c75d77a80497597f99c0b5597ee27'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:47.200638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a6e8f165-4d57-4d55-b713-237366f007c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b1c75d77a80497597f99c0b5597ee27'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:47.213097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a6e8f165-4d57-4d55-b713-237366f007c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b1c75d77a80497597f99c0b5597ee27'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:47.214341 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a6e8f165-4d57-4d55-b713-237366f007c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b1c75d77a80497597f99c0b5597ee27'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:47.215365 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a6e8f165-4d57-4d55-b713-237366f007c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b1c75d77a80497597f99c0b5597ee27'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:47.229147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:47.241302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1274/2539] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:36:47 2022] DELETE /identity/v3/users/2b1c75d77a80497597f99c0b5597ee27 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:47.246068 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef30252-6aba-4c6b-9d33-33ca03b922a1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:47.246673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef30252-6aba-4c6b-9d33-33ca03b922a1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:47.246673 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef30252-6aba-4c6b-9d33-33ca03b922a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:47.271344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4ef30252-6aba-4c6b-9d33-33ca03b922a1 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:47.272563 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4ef30252-6aba-4c6b-9d33-33ca03b922a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:47.339254 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1266/2540] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:47.351973 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8f165-4d57-4d55-b713-237366f007c1 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:47.353478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a6e8f165-4d57-4d55-b713-237366f007c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:47.454677 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8f165-4d57-4d55-b713-237366f007c1 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:47.459313 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98587fea-41cf-48d0-a6ff-b7a2f4af8689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:47.459926 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98587fea-41cf-48d0-a6ff-b7a2f4af8689 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:47.460214 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98587fea-41cf-48d0-a6ff-b7a2f4af8689 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:47.460612 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98587fea-41cf-48d0-a6ff-b7a2f4af8689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:47.470676 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98587fea-41cf-48d0-a6ff-b7a2f4af8689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:47.472179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98587fea-41cf-48d0-a6ff-b7a2f4af8689 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:47.483263 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1275/2541] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:36:47.784504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef30252-6aba-4c6b-9d33-33ca03b922a1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:47.786693 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4ef30252-6aba-4c6b-9d33-33ca03b922a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:47.816460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef30252-6aba-4c6b-9d33-33ca03b922a1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:47.836902 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73bc2ce-2d93-4355-93da-8dd210f4619d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:47.837679 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73bc2ce-2d93-4355-93da-8dd210f4619d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:47.838085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73bc2ce-2d93-4355-93da-8dd210f4619d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:47.838592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73bc2ce-2d93-4355-93da-8dd210f4619d admin admin] PATH_INFO: `/v3/projects/53c1b22f7ffe44dc939cab41ce14e321` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:47.839759 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73bc2ce-2d93-4355-93da-8dd210f4619d admin admin] RBAC: Authorizing `identity:delete_project(project_id=53c1b22f7ffe44dc939cab41ce14e321)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:47.841991 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73bc2ce-2d93-4355-93da-8dd210f4619d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:47.903682 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a73bc2ce-2d93-4355-93da-8dd210f4619d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53c1b22f7ffe44dc939cab41ce14e321', 'request_id': 'req-a73bc2ce-2d93-4355-93da-8dd210f4619d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:47.912095 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a73bc2ce-2d93-4355-93da-8dd210f4619d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53c1b22f7ffe44dc939cab41ce14e321', 'request_id': 'req-a73bc2ce-2d93-4355-93da-8dd210f4619d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:47.923618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a73bc2ce-2d93-4355-93da-8dd210f4619d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53c1b22f7ffe44dc939cab41ce14e321', 'request_id': 'req-a73bc2ce-2d93-4355-93da-8dd210f4619d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:47.938596 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a73bc2ce-2d93-4355-93da-8dd210f4619d admin admin] The token cache is being invalidate because project 53c1b22f7ffe44dc939cab41ce14e321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:36:47.938884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a73bc2ce-2d93-4355-93da-8dd210f4619d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:47.939533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a73bc2ce-2d93-4355-93da-8dd210f4619d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:47.941772 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1267/2542] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:36:47 2022] DELETE /identity/v3/projects/53c1b22f7ffe44dc939cab41ce14e321 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:47.945932 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98587fea-41cf-48d0-a6ff-b7a2f4af8689 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:47.956060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98587fea-41cf-48d0-a6ff-b7a2f4af8689 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:48.036851 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a9c4a-3ae0-4712-8098-909d04976feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:48.037563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a9c4a-3ae0-4712-8098-909d04976feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:48.037750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a9c4a-3ae0-4712-8098-909d04976feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:48.038026 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a9c4a-3ae0-4712-8098-909d04976feb admin admin] PATH_INFO: `/v3/users/6fde2c155d504055918e4a46a1fc1503` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:48.039204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a9c4a-3ae0-4712-8098-909d04976feb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fde2c155d504055918e4a46a1fc1503)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:48.041128 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a9c4a-3ae0-4712-8098-909d04976feb admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:48.089873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-543a9c4a-3ae0-4712-8098-909d04976feb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fde2c155d504055918e4a46a1fc1503'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:48.097553 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-543a9c4a-3ae0-4712-8098-909d04976feb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fde2c155d504055918e4a46a1fc1503'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:48.105429 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-543a9c4a-3ae0-4712-8098-909d04976feb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fde2c155d504055918e4a46a1fc1503'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:48.106197 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-543a9c4a-3ae0-4712-8098-909d04976feb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fde2c155d504055918e4a46a1fc1503'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:48.106999 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-543a9c4a-3ae0-4712-8098-909d04976feb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fde2c155d504055918e4a46a1fc1503'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:48.135775 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1276/2543] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:36:47 2022] DELETE /identity/v3/users/6fde2c155d504055918e4a46a1fc1503 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:48.535186 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73bc2ce-2d93-4355-93da-8dd210f4619d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:48.539384 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a73bc2ce-2d93-4355-93da-8dd210f4619d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:48.561357 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73bc2ce-2d93-4355-93da-8dd210f4619d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:48.573712 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:48.646186 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8206f6-a28e-4f14-9af7-287b3d664775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:48.646867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8206f6-a28e-4f14-9af7-287b3d664775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:48.647150 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8206f6-a28e-4f14-9af7-287b3d664775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:48.647420 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8206f6-a28e-4f14-9af7-287b3d664775 admin admin] PATH_INFO: `/v3/projects/11673bc0bf7e4a4c86151b5bddfd1e8f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:48.648574 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8206f6-a28e-4f14-9af7-287b3d664775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11673bc0bf7e4a4c86151b5bddfd1e8f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:48.650549 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8206f6-a28e-4f14-9af7-287b3d664775 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:48.705710 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d8206f6-a28e-4f14-9af7-287b3d664775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11673bc0bf7e4a4c86151b5bddfd1e8f', 'request_id': 'req-9d8206f6-a28e-4f14-9af7-287b3d664775'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:48.712709 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d8206f6-a28e-4f14-9af7-287b3d664775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11673bc0bf7e4a4c86151b5bddfd1e8f', 'request_id': 'req-9d8206f6-a28e-4f14-9af7-287b3d664775'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:48.721406 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d8206f6-a28e-4f14-9af7-287b3d664775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11673bc0bf7e4a4c86151b5bddfd1e8f', 'request_id': 'req-9d8206f6-a28e-4f14-9af7-287b3d664775'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:48.734649 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d8206f6-a28e-4f14-9af7-287b3d664775 admin admin] The token cache is being invalidate because project 11673bc0bf7e4a4c86151b5bddfd1e8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:36:48.734939 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d8206f6-a28e-4f14-9af7-287b3d664775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:48.735595 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d8206f6-a28e-4f14-9af7-287b3d664775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:48.737074 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1268/2544] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:36:48 2022] DELETE /identity/v3/projects/11673bc0bf7e4a4c86151b5bddfd1e8f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:48.742724 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d05e3-45e5-4f8a-9df0-c8694844c04f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:48.742968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d05e3-45e5-4f8a-9df0-c8694844c04f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:48.743241 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d05e3-45e5-4f8a-9df0-c8694844c04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:48.762741 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f12d05e3-45e5-4f8a-9df0-c8694844c04f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:48.763992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f12d05e3-45e5-4f8a-9df0-c8694844c04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:48.774383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:48.829729 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1277/2545] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:48.837830 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a14a7-d798-4fbb-a41c-87b83be39f23 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:48.838113 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a14a7-d798-4fbb-a41c-87b83be39f23 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:48.838436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a14a7-d798-4fbb-a41c-87b83be39f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:48.860116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b98a14a7-d798-4fbb-a41c-87b83be39f23 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:48.861321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b98a14a7-d798-4fbb-a41c-87b83be39f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:48.870130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1269/2546] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:48.874388 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d05e3-45e5-4f8a-9df0-c8694844c04f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:48.884183 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d05e3-45e5-4f8a-9df0-c8694844c04f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:48.889495 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22f73a-0afb-442f-b638-3fcd3fabf1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:48.890142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22f73a-0afb-442f-b638-3fcd3fabf1da admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:48.890589 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22f73a-0afb-442f-b638-3fcd3fabf1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:48.890864 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22f73a-0afb-442f-b638-3fcd3fabf1da admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:48.891507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22f73a-0afb-442f-b638-3fcd3fabf1da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:36:48.891813 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22f73a-0afb-442f-b638-3fcd3fabf1da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:48.893446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22f73a-0afb-442f-b638-3fcd3fabf1da admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:48.912381 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a14a7-d798-4fbb-a41c-87b83be39f23 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:48.914399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1278/2547] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:36:48.922551 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a14a7-d798-4fbb-a41c-87b83be39f23 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:48.927677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963fadc1-c909-4192-b95d-066734f463fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:48.928453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963fadc1-c909-4192-b95d-066734f463fd admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:48.928792 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963fadc1-c909-4192-b95d-066734f463fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:48.929125 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963fadc1-c909-4192-b95d-066734f463fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:48.929936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963fadc1-c909-4192-b95d-066734f463fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:48.932001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963fadc1-c909-4192-b95d-066734f463fd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:48.963371 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1270/2548] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:36:48 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:36:48.967770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22f73a-0afb-442f-b638-3fcd3fabf1da admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:48.979281 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22f73a-0afb-442f-b638-3fcd3fabf1da admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:49.054052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7590a-219b-4572-939a-89297fc52b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:49.054885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7590a-219b-4572-939a-89297fc52b5e admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:49.054994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7590a-219b-4572-939a-89297fc52b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:49.055232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7590a-219b-4572-939a-89297fc52b5e admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:49.058982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7590a-219b-4572-939a-89297fc52b5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:49.061068 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7590a-219b-4572-939a-89297fc52b5e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:49.086849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1279/2549] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:36:48 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:36:49.091229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963fadc1-c909-4192-b95d-066734f463fd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:49.103173 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963fadc1-c909-4192-b95d-066734f463fd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:49.107834 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182af6b-40c8-4dda-b67e-5de244689999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:49.108418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182af6b-40c8-4dda-b67e-5de244689999 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:49.108696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182af6b-40c8-4dda-b67e-5de244689999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:49.108965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182af6b-40c8-4dda-b67e-5de244689999 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:49.109610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182af6b-40c8-4dda-b67e-5de244689999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:49.111242 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182af6b-40c8-4dda-b67e-5de244689999 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:49.122356 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1271/2550] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:36:49.126091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7590a-219b-4572-939a-89297fc52b5e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:49.136857 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7590a-219b-4572-939a-89297fc52b5e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:49.142026 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad64a7-b3dc-4634-853a-56a6de92e760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:49.142712 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad64a7-b3dc-4634-853a-56a6de92e760 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:49.142981 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad64a7-b3dc-4634-853a-56a6de92e760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:49.143277 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad64a7-b3dc-4634-853a-56a6de92e760 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:49.144459 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad64a7-b3dc-4634-853a-56a6de92e760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:49.146075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad64a7-b3dc-4634-853a-56a6de92e760 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:49.159095 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1280/2551] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:36:49.163721 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182af6b-40c8-4dda-b67e-5de244689999 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:49.174780 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182af6b-40c8-4dda-b67e-5de244689999 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:49.179560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ef11a-c818-4699-b42a-f1ed04b84ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:49.180209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ef11a-c818-4699-b42a-f1ed04b84ace admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:49.180757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ef11a-c818-4699-b42a-f1ed04b84ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:49.181191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ef11a-c818-4699-b42a-f1ed04b84ace admin admin] PATH_INFO: `/v3/projects/35e8b649676048858fafd3ff5af09b6b/users/a057eaac10e8406e85cfef43a87be865/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:49.197403 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ef11a-c818-4699-b42a-f1ed04b84ace admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e8b649676048858fafd3ff5af09b6b, user_id=a057eaac10e8406e85cfef43a87be865, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:49.200234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ef11a-c818-4699-b42a-f1ed04b84ace admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:49.221720 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1272/2552] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:36:49 2022] PUT /identity/v3/projects/35e8b649676048858fafd3ff5af09b6b/users/a057eaac10e8406e85cfef43a87be865/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:36:49.232812 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e285f2-5b6c-43c3-a751-e9560ba43486 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:49.233212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e285f2-5b6c-43c3-a751-e9560ba43486 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:49.233691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e285f2-5b6c-43c3-a751-e9560ba43486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:49.269960 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a7e285f2-5b6c-43c3-a751-e9560ba43486 None None] MFA Rules not processed for user `a057eaac10e8406e85cfef43a87be865`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:49.271245 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a7e285f2-5b6c-43c3-a751-e9560ba43486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:49.337672 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1281/2553] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:36:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:36:49.350016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ef11a-c818-4699-b42a-f1ed04b84ace admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:49.351783 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dd4ef11a-c818-4699-b42a-f1ed04b84ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:49.397203 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:49.430190 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ef11a-c818-4699-b42a-f1ed04b84ace admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:49.434447 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32cf4c4-e874-432f-95a9-9710786ba132 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:49.435267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cf4c4-e874-432f-95a9-9710786ba132 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:49.438363 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cf4c4-e874-432f-95a9-9710786ba132 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:49.438363 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cf4c4-e874-432f-95a9-9710786ba132 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:49.453881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32cf4c4-e874-432f-95a9-9710786ba132 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:49.457296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32cf4c4-e874-432f-95a9-9710786ba132 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:49.468519 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1273/2554] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:36:59.334688 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:59.464010 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=10 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:15.018663 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e285f2-5b6c-43c3-a751-e9560ba43486 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:15.020709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a7e285f2-5b6c-43c3-a751-e9560ba43486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:15.117537 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e285f2-5b6c-43c3-a751-e9560ba43486 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:15.123992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39034230-6f10-447f-990c-69dbabd7e599 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:15.124715 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39034230-6f10-447f-990c-69dbabd7e599 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:15.125058 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39034230-6f10-447f-990c-69dbabd7e599 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:15.125432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39034230-6f10-447f-990c-69dbabd7e599 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:15.127342 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-39034230-6f10-447f-990c-69dbabd7e599 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:15.141322 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39034230-6f10-447f-990c-69dbabd7e599 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:15.143086 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39034230-6f10-447f-990c-69dbabd7e599 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:15.158381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1282/2555] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:16.520088 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32cf4c4-e874-432f-95a9-9710786ba132 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:16.582611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32cf4c4-e874-432f-95a9-9710786ba132 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:16.587774 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d508c-b2d9-45ae-95b5-14c35eee72f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:16.588393 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d508c-b2d9-45ae-95b5-14c35eee72f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:16.588671 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d508c-b2d9-45ae-95b5-14c35eee72f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:16.588941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d508c-b2d9-45ae-95b5-14c35eee72f1 admin admin] PATH_INFO: `/v3/users/a057eaac10e8406e85cfef43a87be865` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:16.590058 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d508c-b2d9-45ae-95b5-14c35eee72f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a057eaac10e8406e85cfef43a87be865)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:16.592037 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d508c-b2d9-45ae-95b5-14c35eee72f1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:16.641011 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b55d508c-b2d9-45ae-95b5-14c35eee72f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a057eaac10e8406e85cfef43a87be865'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:16.650324 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b55d508c-b2d9-45ae-95b5-14c35eee72f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a057eaac10e8406e85cfef43a87be865'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:16.658444 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b55d508c-b2d9-45ae-95b5-14c35eee72f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a057eaac10e8406e85cfef43a87be865'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:16.659233 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b55d508c-b2d9-45ae-95b5-14c35eee72f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a057eaac10e8406e85cfef43a87be865'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:16.659915 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b55d508c-b2d9-45ae-95b5-14c35eee72f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a057eaac10e8406e85cfef43a87be865'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:16.679600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1274/2556] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:37:16 2022] DELETE /identity/v3/users/a057eaac10e8406e85cfef43a87be865 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:16.684039 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda5059-398e-4eb6-9e62-ac4da045063c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:16.684543 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda5059-398e-4eb6-9e62-ac4da045063c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:16.684636 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda5059-398e-4eb6-9e62-ac4da045063c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:16.704690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6bda5059-398e-4eb6-9e62-ac4da045063c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:16.706021 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6bda5059-398e-4eb6-9e62-ac4da045063c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:16.769400 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1283/2557] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:16.779697 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d508c-b2d9-45ae-95b5-14c35eee72f1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:16.781115 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b55d508c-b2d9-45ae-95b5-14c35eee72f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:16.838351 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d508c-b2d9-45ae-95b5-14c35eee72f1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:16.841949 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4af21-18e1-488a-b53f-98568367b3e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:16.842592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4af21-18e1-488a-b53f-98568367b3e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:16.842868 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4af21-18e1-488a-b53f-98568367b3e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:16.843138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4af21-18e1-488a-b53f-98568367b3e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:16.852739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4af21-18e1-488a-b53f-98568367b3e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:16.854168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4af21-18e1-488a-b53f-98568367b3e8 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:16.863455 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1275/2558] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:17.284821 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda5059-398e-4eb6-9e62-ac4da045063c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:17.286330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6bda5059-398e-4eb6-9e62-ac4da045063c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:17.298191 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda5059-398e-4eb6-9e62-ac4da045063c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:17.303774 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8707c51-8ac0-4d74-959f-8059eded8ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:17.304350 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8707c51-8ac0-4d74-959f-8059eded8ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:17.304624 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8707c51-8ac0-4d74-959f-8059eded8ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:17.304894 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8707c51-8ac0-4d74-959f-8059eded8ab8 admin admin] PATH_INFO: `/v3/projects/35e8b649676048858fafd3ff5af09b6b` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:17.306499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8707c51-8ac0-4d74-959f-8059eded8ab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35e8b649676048858fafd3ff5af09b6b)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:17.308110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8707c51-8ac0-4d74-959f-8059eded8ab8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:17.370087 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e8707c51-8ac0-4d74-959f-8059eded8ab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35e8b649676048858fafd3ff5af09b6b', 'request_id': 'req-e8707c51-8ac0-4d74-959f-8059eded8ab8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:17.376297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e8707c51-8ac0-4d74-959f-8059eded8ab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35e8b649676048858fafd3ff5af09b6b', 'request_id': 'req-e8707c51-8ac0-4d74-959f-8059eded8ab8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:17.388181 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e8707c51-8ac0-4d74-959f-8059eded8ab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35e8b649676048858fafd3ff5af09b6b', 'request_id': 'req-e8707c51-8ac0-4d74-959f-8059eded8ab8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:17.403701 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e8707c51-8ac0-4d74-959f-8059eded8ab8 admin admin] The token cache is being invalidate because project 35e8b649676048858fafd3ff5af09b6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:37:17.403701 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e8707c51-8ac0-4d74-959f-8059eded8ab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:17.404535 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e8707c51-8ac0-4d74-959f-8059eded8ab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:17.406302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1284/2559] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:37:17 2022] DELETE /identity/v3/projects/35e8b649676048858fafd3ff5af09b6b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:17.411825 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ae1b0-aec8-4dc7-9805-8e442a89c8ab None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:17.412216 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ae1b0-aec8-4dc7-9805-8e442a89c8ab None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:17.413690 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ae1b0-aec8-4dc7-9805-8e442a89c8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:17.448462 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-297ae1b0-aec8-4dc7-9805-8e442a89c8ab None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:17.449833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-297ae1b0-aec8-4dc7-9805-8e442a89c8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:17.476327 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:17.523996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1276/2560] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:17.592235 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e71e-7abf-428f-841e-42596a3c852c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:17.592665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e71e-7abf-428f-841e-42596a3c852c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:17.593418 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e71e-7abf-428f-841e-42596a3c852c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:17.613586 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-61c5e71e-7abf-428f-841e-42596a3c852c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:17.614844 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-61c5e71e-7abf-428f-841e-42596a3c852c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:17.623377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1285/2561] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:17.627600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ae1b0-aec8-4dc7-9805-8e442a89c8ab None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:17.638070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ae1b0-aec8-4dc7-9805-8e442a89c8ab None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:17.642794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f4863-12aa-45ff-a4e4-3c57350999fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:17.643420 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f4863-12aa-45ff-a4e4-3c57350999fa admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:17.643702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f4863-12aa-45ff-a4e4-3c57350999fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:17.644033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f4863-12aa-45ff-a4e4-3c57350999fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:17.644722 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f4863-12aa-45ff-a4e4-3c57350999fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:37:17.645058 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f4863-12aa-45ff-a4e4-3c57350999fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:17.646695 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f4863-12aa-45ff-a4e4-3c57350999fa admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:17.658646 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1277/2562] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:37:17.664229 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e71e-7abf-428f-841e-42596a3c852c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:17.672916 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:17.674344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e71e-7abf-428f-841e-42596a3c852c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:17.681864 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4880c11-bdc3-465c-8691-e4b354a454a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:17.682722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4880c11-bdc3-465c-8691-e4b354a454a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:17.683014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4880c11-bdc3-465c-8691-e4b354a454a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:17.683344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4880c11-bdc3-465c-8691-e4b354a454a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:17.684107 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4880c11-bdc3-465c-8691-e4b354a454a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:17.686338 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4880c11-bdc3-465c-8691-e4b354a454a6 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:17.716595 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1286/2563] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:37:17 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:37:17.720818 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f4863-12aa-45ff-a4e4-3c57350999fa admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:17.732870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f4863-12aa-45ff-a4e4-3c57350999fa admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:17.802556 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae5778-8865-4a2f-ac89-82fc21321d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:17.804058 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae5778-8865-4a2f-ac89-82fc21321d01 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:17.804342 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae5778-8865-4a2f-ac89-82fc21321d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:17.804614 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae5778-8865-4a2f-ac89-82fc21321d01 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:17.805241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae5778-8865-4a2f-ac89-82fc21321d01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:17.807205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae5778-8865-4a2f-ac89-82fc21321d01 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:17.831122 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1278/2564] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:37:17 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:37:17.835110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4880c11-bdc3-465c-8691-e4b354a454a6 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:17.845779 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4880c11-bdc3-465c-8691-e4b354a454a6 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:17.851179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13144493-84ea-4f4c-b8fa-f8e81bc7906e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:17.851777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13144493-84ea-4f4c-b8fa-f8e81bc7906e admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:17.852056 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13144493-84ea-4f4c-b8fa-f8e81bc7906e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:17.852325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13144493-84ea-4f4c-b8fa-f8e81bc7906e admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:17.852976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13144493-84ea-4f4c-b8fa-f8e81bc7906e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:17.854591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13144493-84ea-4f4c-b8fa-f8e81bc7906e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:17.864971 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1287/2565] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:37:17.869789 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae5778-8865-4a2f-ac89-82fc21321d01 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:17.880667 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae5778-8865-4a2f-ac89-82fc21321d01 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:17.885503 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169ae3c-4dc9-4d49-95a3-30e897f23087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:17.886092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169ae3c-4dc9-4d49-95a3-30e897f23087 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:17.886408 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169ae3c-4dc9-4d49-95a3-30e897f23087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:17.886688 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169ae3c-4dc9-4d49-95a3-30e897f23087 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:17.887331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169ae3c-4dc9-4d49-95a3-30e897f23087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:17.888963 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169ae3c-4dc9-4d49-95a3-30e897f23087 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:17.900414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1279/2566] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:37:17.904512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13144493-84ea-4f4c-b8fa-f8e81bc7906e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:17.915140 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13144493-84ea-4f4c-b8fa-f8e81bc7906e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:17.919741 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a13b54-b2d7-4fe5-9628-910360fa83d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:17.920369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a13b54-b2d7-4fe5-9628-910360fa83d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:17.920643 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a13b54-b2d7-4fe5-9628-910360fa83d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:17.920909 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a13b54-b2d7-4fe5-9628-910360fa83d6 admin admin] PATH_INFO: `/v3/projects/2aa5a07ea36f4d87a30b3f1c9a401cf5/users/3d616898452a49c8ae0f73698603c942/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:17.935760 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a13b54-b2d7-4fe5-9628-910360fa83d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa5a07ea36f4d87a30b3f1c9a401cf5, user_id=3d616898452a49c8ae0f73698603c942, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:17.938394 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a13b54-b2d7-4fe5-9628-910360fa83d6 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:17.951128 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1288/2567] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:37:17 2022] PUT /identity/v3/projects/2aa5a07ea36f4d87a30b3f1c9a401cf5/users/3d616898452a49c8ae0f73698603c942/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:17.960972 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38b749-af80-47c9-b9c4-8c3e0c9a1228 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:17.961265 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38b749-af80-47c9-b9c4-8c3e0c9a1228 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:17.961606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38b749-af80-47c9-b9c4-8c3e0c9a1228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:17.989061 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ef38b749-af80-47c9-b9c4-8c3e0c9a1228 None None] MFA Rules not processed for user `3d616898452a49c8ae0f73698603c942`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:17.990926 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ef38b749-af80-47c9-b9c4-8c3e0c9a1228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:18.054024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1280/2568] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:18.065491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a13b54-b2d7-4fe5-9628-910360fa83d6 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:18.067220 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-99a13b54-b2d7-4fe5-9628-910360fa83d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:18.126491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a13b54-b2d7-4fe5-9628-910360fa83d6 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:18.130644 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179d9c8-ab69-4fcc-ae15-0eab52dcf4ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:18.131297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179d9c8-ab69-4fcc-ae15-0eab52dcf4ef service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:18.131538 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179d9c8-ab69-4fcc-ae15-0eab52dcf4ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:18.131858 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179d9c8-ab69-4fcc-ae15-0eab52dcf4ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:18.140985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179d9c8-ab69-4fcc-ae15-0eab52dcf4ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:18.142436 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179d9c8-ab69-4fcc-ae15-0eab52dcf4ef service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:18.151661 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1289/2569] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:18.284422 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be225dfd-b48f-49ff-8225-910e0c437f49 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:18.284705 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be225dfd-b48f-49ff-8225-910e0c437f49 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:18.284998 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be225dfd-b48f-49ff-8225-910e0c437f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:18.307327 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-be225dfd-b48f-49ff-8225-910e0c437f49 None None] MFA Rules not processed for user `b88ccd3e5c4c48d3a27f4eaa7874e28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:18.308987 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-be225dfd-b48f-49ff-8225-910e0c437f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:18.323801 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:18.378119 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1281/2570] 10.222.0.39 () {58 vars in 967 bytes} [Mon Aug 8 02:37:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:18.395231 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179d9c8-ab69-4fcc-ae15-0eab52dcf4ef service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:18.397406 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6179d9c8-ab69-4fcc-ae15-0eab52dcf4ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:18.499119 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179d9c8-ab69-4fcc-ae15-0eab52dcf4ef service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:18.504322 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9861e-c4e6-47c4-8802-286ce00cc832 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac60b4d621234225830c94edfac400e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:18.504967 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9861e-c4e6-47c4-8802-286ce00cc832 service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:18.505247 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9861e-c4e6-47c4-8802-286ce00cc832 service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:18.505520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9861e-c4e6-47c4-8802-286ce00cc832 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:18.514590 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9861e-c4e6-47c4-8802-286ce00cc832 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:18.516169 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9861e-c4e6-47c4-8802-286ce00cc832 service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:18.525560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1290/2571] 10.222.0.39 () {60 vars in 1384 bytes} [Mon Aug 8 02:37:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:37:21.198995 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be225dfd-b48f-49ff-8225-910e0c437f49 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:21.261648 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be225dfd-b48f-49ff-8225-910e0c437f49 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:21.266133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0883b7-3175-402e-a2d5-4ac4d9b9cbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:21.266767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0883b7-3175-402e-a2d5-4ac4d9b9cbae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:21.267041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0883b7-3175-402e-a2d5-4ac4d9b9cbae admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:21.267308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0883b7-3175-402e-a2d5-4ac4d9b9cbae admin admin] PATH_INFO: `/v3/users/3d616898452a49c8ae0f73698603c942` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:21.268410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0883b7-3175-402e-a2d5-4ac4d9b9cbae admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d616898452a49c8ae0f73698603c942)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:21.270355 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0883b7-3175-402e-a2d5-4ac4d9b9cbae admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:21.314198 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-be0883b7-3175-402e-a2d5-4ac4d9b9cbae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d616898452a49c8ae0f73698603c942'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:21.322573 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-be0883b7-3175-402e-a2d5-4ac4d9b9cbae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d616898452a49c8ae0f73698603c942'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:21.329742 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-be0883b7-3175-402e-a2d5-4ac4d9b9cbae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d616898452a49c8ae0f73698603c942'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:21.330537 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-be0883b7-3175-402e-a2d5-4ac4d9b9cbae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d616898452a49c8ae0f73698603c942'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:21.331183 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-be0883b7-3175-402e-a2d5-4ac4d9b9cbae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d616898452a49c8ae0f73698603c942'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:21.350627 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1282/2572] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:37:21 2022] DELETE /identity/v3/users/3d616898452a49c8ae0f73698603c942 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:21.355167 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152303a-b534-4e80-8ea0-dd592b03518d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:21.355662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152303a-b534-4e80-8ea0-dd592b03518d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:21.355873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152303a-b534-4e80-8ea0-dd592b03518d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:21.380052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-1152303a-b534-4e80-8ea0-dd592b03518d None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:21.381286 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1152303a-b534-4e80-8ea0-dd592b03518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:21.406854 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0883b7-3175-402e-a2d5-4ac4d9b9cbae admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:21.408671 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-be0883b7-3175-402e-a2d5-4ac4d9b9cbae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:21.418390 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:21.447432 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1291/2573] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:21.457499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152303a-b534-4e80-8ea0-dd592b03518d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:21.459384 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1152303a-b534-4e80-8ea0-dd592b03518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:21.461721 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:21.474279 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0883b7-3175-402e-a2d5-4ac4d9b9cbae admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:21.478813 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa1d28-5fd4-48c2-bf09-3b367864d575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:21.479412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa1d28-5fd4-48c2-bf09-3b367864d575 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:21.479689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa1d28-5fd4-48c2-bf09-3b367864d575 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:21.479959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa1d28-5fd4-48c2-bf09-3b367864d575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:21.481666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a8aa1d28-5fd4-48c2-bf09-3b367864d575 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:21.493955 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa1d28-5fd4-48c2-bf09-3b367864d575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:21.495432 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa1d28-5fd4-48c2-bf09-3b367864d575 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:21.505362 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1283/2574] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:37:21.517910 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152303a-b534-4e80-8ea0-dd592b03518d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:21.522323 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddb488-90e8-4fd5-86d6-176acbd1a614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:21.522934 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddb488-90e8-4fd5-86d6-176acbd1a614 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:21.523212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddb488-90e8-4fd5-86d6-176acbd1a614 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:21.523483 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddb488-90e8-4fd5-86d6-176acbd1a614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:21.532757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ddb488-90e8-4fd5-86d6-176acbd1a614 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:21.534192 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ddb488-90e8-4fd5-86d6-176acbd1a614 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:21.543626 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1292/2575] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:37:22.471045 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa1d28-5fd4-48c2-bf09-3b367864d575 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:22.472509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a8aa1d28-5fd4-48c2-bf09-3b367864d575 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:22.487930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa1d28-5fd4-48c2-bf09-3b367864d575 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:22.496119 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a9282-3080-485d-8d36-61103cabd297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:22.497099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a9282-3080-485d-8d36-61103cabd297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:22.497384 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a9282-3080-485d-8d36-61103cabd297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:22.497653 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a9282-3080-485d-8d36-61103cabd297 admin admin] PATH_INFO: `/v3/projects/2aa5a07ea36f4d87a30b3f1c9a401cf5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:22.498851 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a9282-3080-485d-8d36-61103cabd297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aa5a07ea36f4d87a30b3f1c9a401cf5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:22.500726 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a9282-3080-485d-8d36-61103cabd297 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:22.556819 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b62a9282-3080-485d-8d36-61103cabd297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aa5a07ea36f4d87a30b3f1c9a401cf5', 'request_id': 'req-b62a9282-3080-485d-8d36-61103cabd297'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:22.562578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b62a9282-3080-485d-8d36-61103cabd297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aa5a07ea36f4d87a30b3f1c9a401cf5', 'request_id': 'req-b62a9282-3080-485d-8d36-61103cabd297'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:22.570085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b62a9282-3080-485d-8d36-61103cabd297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aa5a07ea36f4d87a30b3f1c9a401cf5', 'request_id': 'req-b62a9282-3080-485d-8d36-61103cabd297'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:22.582758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b62a9282-3080-485d-8d36-61103cabd297 admin admin] The token cache is being invalidate because project 2aa5a07ea36f4d87a30b3f1c9a401cf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:37:22.583042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b62a9282-3080-485d-8d36-61103cabd297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:22.583677 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b62a9282-3080-485d-8d36-61103cabd297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:22.585228 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1284/2576] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:37:22 2022] DELETE /identity/v3/projects/2aa5a07ea36f4d87a30b3f1c9a401cf5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:22.590297 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf9f21-7d75-47df-829d-4f9949213e27 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:22.590816 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf9f21-7d75-47df-829d-4f9949213e27 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:22.590816 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf9f21-7d75-47df-829d-4f9949213e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:22.610124 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-1dcf9f21-7d75-47df-829d-4f9949213e27 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:22.611278 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1dcf9f21-7d75-47df-829d-4f9949213e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:22.672678 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1293/2577] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:22.680135 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b0362-9d3b-4c9b-9494-cc830918127f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:22.680420 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b0362-9d3b-4c9b-9494-cc830918127f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:22.680691 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b0362-9d3b-4c9b-9494-cc830918127f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:22.700545 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9e8b0362-9d3b-4c9b-9494-cc830918127f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:22.701757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9e8b0362-9d3b-4c9b-9494-cc830918127f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:22.710653 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1285/2578] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:37:22.714732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf9f21-7d75-47df-829d-4f9949213e27 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:22.727596 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf9f21-7d75-47df-829d-4f9949213e27 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:22.732302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2308bd-98ce-43c1-8420-fc3b116e5468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:22.732951 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2308bd-98ce-43c1-8420-fc3b116e5468 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:22.733242 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2308bd-98ce-43c1-8420-fc3b116e5468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:22.733514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2308bd-98ce-43c1-8420-fc3b116e5468 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:22.734165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2308bd-98ce-43c1-8420-fc3b116e5468 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:37:22.734502 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2308bd-98ce-43c1-8420-fc3b116e5468 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:22.736098 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2308bd-98ce-43c1-8420-fc3b116e5468 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:22.752685 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1294/2579] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:37:22.753288 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b0362-9d3b-4c9b-9494-cc830918127f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:22.766482 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b0362-9d3b-4c9b-9494-cc830918127f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:22.771405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32efe3-ee28-4e84-b16d-7108943931de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:22.772013 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32efe3-ee28-4e84-b16d-7108943931de admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:22.772289 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32efe3-ee28-4e84-b16d-7108943931de admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:22.772595 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32efe3-ee28-4e84-b16d-7108943931de admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:22.773203 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32efe3-ee28-4e84-b16d-7108943931de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:22.775191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32efe3-ee28-4e84-b16d-7108943931de admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:22.808674 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1286/2580] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:37:22 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:37:22.812516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2308bd-98ce-43c1-8420-fc3b116e5468 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:22.823897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2308bd-98ce-43c1-8420-fc3b116e5468 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:22.887802 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94137a-e819-4a8e-8832-9511fa14b95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:22.888417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94137a-e819-4a8e-8832-9511fa14b95a admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:22.890302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94137a-e819-4a8e-8832-9511fa14b95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:22.890302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94137a-e819-4a8e-8832-9511fa14b95a admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:22.890881 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94137a-e819-4a8e-8832-9511fa14b95a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:22.892933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94137a-e819-4a8e-8832-9511fa14b95a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:22.912834 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:22.920408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1295/2581] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:37:22 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:37:22.924179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32efe3-ee28-4e84-b16d-7108943931de admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:22.936619 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32efe3-ee28-4e84-b16d-7108943931de admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:22.942145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3613a99-dfdf-4013-8372-0126d717e9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:22.942794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3613a99-dfdf-4013-8372-0126d717e9ab admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:22.943068 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3613a99-dfdf-4013-8372-0126d717e9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:22.943994 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3613a99-dfdf-4013-8372-0126d717e9ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:22.944637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3613a99-dfdf-4013-8372-0126d717e9ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:22.946277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3613a99-dfdf-4013-8372-0126d717e9ab admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:22.960283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1287/2582] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:37:22.961455 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94137a-e819-4a8e-8832-9511fa14b95a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:22.971257 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94137a-e819-4a8e-8832-9511fa14b95a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:22.976087 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88d997-afdc-426c-93b8-59b3f371ac6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:22.976680 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88d997-afdc-426c-93b8-59b3f371ac6b admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:22.976957 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88d997-afdc-426c-93b8-59b3f371ac6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:22.977224 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88d997-afdc-426c-93b8-59b3f371ac6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:22.977859 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88d997-afdc-426c-93b8-59b3f371ac6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:22.979546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88d997-afdc-426c-93b8-59b3f371ac6b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:22.993406 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1296/2583] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:37:22.994540 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3613a99-dfdf-4013-8372-0126d717e9ab admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:23.004508 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3613a99-dfdf-4013-8372-0126d717e9ab admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:23.009662 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346da5b2-f049-4217-8ea7-d2ee0a6e75c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:23.010344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346da5b2-f049-4217-8ea7-d2ee0a6e75c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:23.010624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346da5b2-f049-4217-8ea7-d2ee0a6e75c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:23.010896 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346da5b2-f049-4217-8ea7-d2ee0a6e75c2 admin admin] PATH_INFO: `/v3/projects/7a06a8947bd84ac398bf498d8816717c/users/72485bc9ed6f4684b77b617597f99e29/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:23.026553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346da5b2-f049-4217-8ea7-d2ee0a6e75c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a06a8947bd84ac398bf498d8816717c, user_id=72485bc9ed6f4684b77b617597f99e29, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:23.029212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346da5b2-f049-4217-8ea7-d2ee0a6e75c2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:23.044381 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1288/2584] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:37:22 2022] PUT /identity/v3/projects/7a06a8947bd84ac398bf498d8816717c/users/72485bc9ed6f4684b77b617597f99e29/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:23.051142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3964da8-e940-40eb-9660-2d0fe5d40395 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:23.051481 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3964da8-e940-40eb-9660-2d0fe5d40395 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:23.051759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3964da8-e940-40eb-9660-2d0fe5d40395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:23.071892 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f3964da8-e940-40eb-9660-2d0fe5d40395 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:23.073115 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f3964da8-e940-40eb-9660-2d0fe5d40395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:23.134042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1297/2585] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:37:23.143644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346da5b2-f049-4217-8ea7-d2ee0a6e75c2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:23.145360 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-346da5b2-f049-4217-8ea7-d2ee0a6e75c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:23.202045 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346da5b2-f049-4217-8ea7-d2ee0a6e75c2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:23.205674 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e51bd-950e-4c89-ba59-97a3f8e8e9a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:23.206292 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e51bd-950e-4c89-ba59-97a3f8e8e9a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:23.206570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e51bd-950e-4c89-ba59-97a3f8e8e9a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:23.206836 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e51bd-950e-4c89-ba59-97a3f8e8e9a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:23.215875 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e51bd-950e-4c89-ba59-97a3f8e8e9a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:23.217654 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e51bd-950e-4c89-ba59-97a3f8e8e9a6 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:23.228770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1289/2586] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:25.171729 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3964da8-e940-40eb-9660-2d0fe5d40395 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:25.174308 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f3964da8-e940-40eb-9660-2d0fe5d40395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:25.215595 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e51bd-950e-4c89-ba59-97a3f8e8e9a6 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:25.217546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9d1e51bd-950e-4c89-ba59-97a3f8e8e9a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:25.237843 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e51bd-950e-4c89-ba59-97a3f8e8e9a6 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:25.243042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a886f-0142-4555-ab7f-6ffbae4ebf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:25.243669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a886f-0142-4555-ab7f-6ffbae4ebf81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:25.243950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a886f-0142-4555-ab7f-6ffbae4ebf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:25.244224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a886f-0142-4555-ab7f-6ffbae4ebf81 admin admin] PATH_INFO: `/v3/users/43dd72b161f8442285dea411741d66b3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:25.246379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:25.263517 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a886f-0142-4555-ab7f-6ffbae4ebf81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43dd72b161f8442285dea411741d66b3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:25.266093 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a886f-0142-4555-ab7f-6ffbae4ebf81 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:25.287521 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3964da8-e940-40eb-9660-2d0fe5d40395 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:25.293168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9726294-d6a8-4582-9c00-073cec8484c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f62745b92a864480a9438de581e1d05e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:25.293917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9726294-d6a8-4582-9c00-073cec8484c3 service placement] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:25.294300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9726294-d6a8-4582-9c00-073cec8484c3 service placement] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:25.295955 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9726294-d6a8-4582-9c00-073cec8484c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:25.298007 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b9726294-d6a8-4582-9c00-073cec8484c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:25.313248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9726294-d6a8-4582-9c00-073cec8484c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:25.314921 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9726294-d6a8-4582-9c00-073cec8484c3 service placement] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:25.326383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1298/2587] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:25.331251 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-025a886f-0142-4555-ab7f-6ffbae4ebf81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43dd72b161f8442285dea411741d66b3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:25.341132 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-025a886f-0142-4555-ab7f-6ffbae4ebf81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43dd72b161f8442285dea411741d66b3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:25.349963 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-025a886f-0142-4555-ab7f-6ffbae4ebf81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43dd72b161f8442285dea411741d66b3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:25.350820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-025a886f-0142-4555-ab7f-6ffbae4ebf81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43dd72b161f8442285dea411741d66b3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:25.351846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-025a886f-0142-4555-ab7f-6ffbae4ebf81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43dd72b161f8442285dea411741d66b3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:25.371293 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1290/2588] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:37:25 2022] DELETE /identity/v3/users/43dd72b161f8442285dea411741d66b3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:25.606331 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9726294-d6a8-4582-9c00-073cec8484c3 service placement] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:25.629337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9726294-d6a8-4582-9c00-073cec8484c3 service placement] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:25.715933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28ac29-a8f2-4b8e-9c35-56aaf360eb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:25.716581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28ac29-a8f2-4b8e-9c35-56aaf360eb3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:25.716863 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28ac29-a8f2-4b8e-9c35-56aaf360eb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:25.717152 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28ac29-a8f2-4b8e-9c35-56aaf360eb3a admin admin] PATH_INFO: `/v3/projects/3c533d28fa3f4d11b88b630a7c7bd1b4` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:25.735664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca28ac29-a8f2-4b8e-9c35-56aaf360eb3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c533d28fa3f4d11b88b630a7c7bd1b4)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:25.738072 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca28ac29-a8f2-4b8e-9c35-56aaf360eb3a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:25.751553 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:25.819493 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ca28ac29-a8f2-4b8e-9c35-56aaf360eb3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c533d28fa3f4d11b88b630a7c7bd1b4', 'request_id': 'req-ca28ac29-a8f2-4b8e-9c35-56aaf360eb3a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:25.827914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ca28ac29-a8f2-4b8e-9c35-56aaf360eb3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c533d28fa3f4d11b88b630a7c7bd1b4', 'request_id': 'req-ca28ac29-a8f2-4b8e-9c35-56aaf360eb3a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:25.840780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ca28ac29-a8f2-4b8e-9c35-56aaf360eb3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c533d28fa3f4d11b88b630a7c7bd1b4', 'request_id': 'req-ca28ac29-a8f2-4b8e-9c35-56aaf360eb3a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:25.859063 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ca28ac29-a8f2-4b8e-9c35-56aaf360eb3a admin admin] The token cache is being invalidate because project 3c533d28fa3f4d11b88b630a7c7bd1b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:37:25.859688 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ca28ac29-a8f2-4b8e-9c35-56aaf360eb3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:25.860113 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ca28ac29-a8f2-4b8e-9c35-56aaf360eb3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:25.866338 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1299/2589] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:37:25 2022] DELETE /identity/v3/projects/3c533d28fa3f4d11b88b630a7c7bd1b4 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:25.871497 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8f015-361e-4b9b-8699-5b4cf608eb6f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:25.871878 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8f015-361e-4b9b-8699-5b4cf608eb6f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:25.872304 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8f015-361e-4b9b-8699-5b4cf608eb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:25.894657 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8ad8f015-361e-4b9b-8699-5b4cf608eb6f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:25.897332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8ad8f015-361e-4b9b-8699-5b4cf608eb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:25.971817 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1291/2590] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:25.980638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f98f01-f817-4965-ac6b-d145b6ece96e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:25.980976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f98f01-f817-4965-ac6b-d145b6ece96e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:25.981296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f98f01-f817-4965-ac6b-d145b6ece96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:26.002534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-25f98f01-f817-4965-ac6b-d145b6ece96e None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:26.003766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-25f98f01-f817-4965-ac6b-d145b6ece96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:26.012499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1300/2591] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:37:26.016768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8f015-361e-4b9b-8699-5b4cf608eb6f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:26.027286 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8f015-361e-4b9b-8699-5b4cf608eb6f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:26.032429 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113ffcc-4e13-4ea5-8030-fdae0a1fcc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:26.033144 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113ffcc-4e13-4ea5-8030-fdae0a1fcc4b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:26.033426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113ffcc-4e13-4ea5-8030-fdae0a1fcc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:26.033696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113ffcc-4e13-4ea5-8030-fdae0a1fcc4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:26.034374 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113ffcc-4e13-4ea5-8030-fdae0a1fcc4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:37:26.034682 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113ffcc-4e13-4ea5-8030-fdae0a1fcc4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:26.036290 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113ffcc-4e13-4ea5-8030-fdae0a1fcc4b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:26.050168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1292/2592] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:37:26.055183 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f98f01-f817-4965-ac6b-d145b6ece96e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:26.067368 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f98f01-f817-4965-ac6b-d145b6ece96e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:26.072390 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0548da-0a18-4b59-b5cc-705da3db492c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:26.073194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0548da-0a18-4b59-b5cc-705da3db492c admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:26.073316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0548da-0a18-4b59-b5cc-705da3db492c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:26.073508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0548da-0a18-4b59-b5cc-705da3db492c admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:26.074144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0548da-0a18-4b59-b5cc-705da3db492c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:26.076116 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0548da-0a18-4b59-b5cc-705da3db492c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:26.106902 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1301/2593] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:37:26 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:37:26.111968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113ffcc-4e13-4ea5-8030-fdae0a1fcc4b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:26.125281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113ffcc-4e13-4ea5-8030-fdae0a1fcc4b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:26.186239 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:26.196515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0d258-3d3b-467a-af2e-0212e62c2b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:26.197102 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0d258-3d3b-467a-af2e-0212e62c2b45 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:26.197386 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0d258-3d3b-467a-af2e-0212e62c2b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:26.197655 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0d258-3d3b-467a-af2e-0212e62c2b45 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:26.198365 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0d258-3d3b-467a-af2e-0212e62c2b45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:26.201823 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0d258-3d3b-467a-af2e-0212e62c2b45 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:26.228610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1293/2594] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:37:26 2022] POST /identity/v3/users => generated 290 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:37:26.232830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0548da-0a18-4b59-b5cc-705da3db492c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:26.245186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0548da-0a18-4b59-b5cc-705da3db492c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:26.250213 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef775cf9-09b1-440b-abcd-6c6df0ad01f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:26.250996 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef775cf9-09b1-440b-abcd-6c6df0ad01f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:26.251371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef775cf9-09b1-440b-abcd-6c6df0ad01f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:26.251886 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef775cf9-09b1-440b-abcd-6c6df0ad01f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:26.252641 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef775cf9-09b1-440b-abcd-6c6df0ad01f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:26.254363 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef775cf9-09b1-440b-abcd-6c6df0ad01f7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:26.269620 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1302/2595] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:37:26.274428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0d258-3d3b-467a-af2e-0212e62c2b45 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:26.286175 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0d258-3d3b-467a-af2e-0212e62c2b45 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:26.291966 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7a57c-8b8b-47c8-a9c2-a0577163fb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:26.293020 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7a57c-8b8b-47c8-a9c2-a0577163fb40 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:26.293308 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7a57c-8b8b-47c8-a9c2-a0577163fb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:26.293611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7a57c-8b8b-47c8-a9c2-a0577163fb40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:26.295049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7a57c-8b8b-47c8-a9c2-a0577163fb40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:26.296658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7a57c-8b8b-47c8-a9c2-a0577163fb40 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:26.309213 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1294/2596] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:37:26.313357 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef775cf9-09b1-440b-abcd-6c6df0ad01f7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:26.325646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef775cf9-09b1-440b-abcd-6c6df0ad01f7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:26.331078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efddf9-7205-4db9-b040-4e5fe7850621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:26.331747 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efddf9-7205-4db9-b040-4e5fe7850621 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:26.332014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efddf9-7205-4db9-b040-4e5fe7850621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:26.333290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efddf9-7205-4db9-b040-4e5fe7850621 admin admin] PATH_INFO: `/v3/projects/8602dee834aa40dcbf6fe726494fe7c8/users/79ca99e4bc6b42dabc2cfda9b797a008/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:26.356055 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67efddf9-7205-4db9-b040-4e5fe7850621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8602dee834aa40dcbf6fe726494fe7c8, user_id=79ca99e4bc6b42dabc2cfda9b797a008, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:26.359831 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67efddf9-7205-4db9-b040-4e5fe7850621 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:26.387893 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1303/2597] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:37:26 2022] PUT /identity/v3/projects/8602dee834aa40dcbf6fe726494fe7c8/users/79ca99e4bc6b42dabc2cfda9b797a008/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:37:26.399412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0291589-bb3d-44e4-852f-a29569db2fcd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:26.399984 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0291589-bb3d-44e4-852f-a29569db2fcd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:26.400439 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0291589-bb3d-44e4-852f-a29569db2fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:26.437870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b0291589-bb3d-44e4-852f-a29569db2fcd None None] MFA Rules not processed for user `79ca99e4bc6b42dabc2cfda9b797a008`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:26.439711 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b0291589-bb3d-44e4-852f-a29569db2fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:26.540135 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1295/2598] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:26.544267 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efddf9-7205-4db9-b040-4e5fe7850621 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:26.636159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efddf9-7205-4db9-b040-4e5fe7850621 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:26.646691 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c303d76-f2cb-456f-a352-586989d8865f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:26.648150 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c303d76-f2cb-456f-a352-586989d8865f admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:26.648487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c303d76-f2cb-456f-a352-586989d8865f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:26.648829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c303d76-f2cb-456f-a352-586989d8865f admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:26.650334 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c303d76-f2cb-456f-a352-586989d8865f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:26.653409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c303d76-f2cb-456f-a352-586989d8865f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:26.694425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1304/2599] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:37:26 2022] POST /identity/v3/projects => generated 381 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:37:26.696684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0291589-bb3d-44e4-852f-a29569db2fcd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:26.712511 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0291589-bb3d-44e4-852f-a29569db2fcd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:26.795876 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627df65-e9b2-45f3-a628-61c331bc3ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:26.797407 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627df65-e9b2-45f3-a628-61c331bc3ddc admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:26.797955 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627df65-e9b2-45f3-a628-61c331bc3ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:26.798327 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627df65-e9b2-45f3-a628-61c331bc3ddc admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:26.799326 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627df65-e9b2-45f3-a628-61c331bc3ddc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:26.802184 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627df65-e9b2-45f3-a628-61c331bc3ddc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:26.850770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1296/2600] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:37:26 2022] POST /identity/v3/users => generated 297 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:37:26.854692 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c303d76-f2cb-456f-a352-586989d8865f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:26.868003 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c303d76-f2cb-456f-a352-586989d8865f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:26.873168 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf00740-a731-40e3-a5f7-8b729801ae88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:26.873794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf00740-a731-40e3-a5f7-8b729801ae88 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:26.874073 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf00740-a731-40e3-a5f7-8b729801ae88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:26.875358 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf00740-a731-40e3-a5f7-8b729801ae88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:26.876111 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf00740-a731-40e3-a5f7-8b729801ae88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:26.877900 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf00740-a731-40e3-a5f7-8b729801ae88 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:26.891060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1305/2601] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:37:26.894800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627df65-e9b2-45f3-a628-61c331bc3ddc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:26.906807 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627df65-e9b2-45f3-a628-61c331bc3ddc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:26.913232 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6365a7-5136-4709-aab1-35da3254c43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:26.914008 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6365a7-5136-4709-aab1-35da3254c43b admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:26.915189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6365a7-5136-4709-aab1-35da3254c43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:26.915715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6365a7-5136-4709-aab1-35da3254c43b admin admin] PATH_INFO: `/v3/projects/ebe3c4358dfb4a919b91fd18caff463c/users/8a508d56ad5d43f489e0fc0925eaf704/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:26.937716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6365a7-5136-4709-aab1-35da3254c43b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebe3c4358dfb4a919b91fd18caff463c, user_id=8a508d56ad5d43f489e0fc0925eaf704, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:26.941361 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6365a7-5136-4709-aab1-35da3254c43b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:26.967506 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1297/2602] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:37:26 2022] PUT /identity/v3/projects/ebe3c4358dfb4a919b91fd18caff463c/users/8a508d56ad5d43f489e0fc0925eaf704/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:37:26.969501 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf00740-a731-40e3-a5f7-8b729801ae88 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:26.984512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf00740-a731-40e3-a5f7-8b729801ae88 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:26.991642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:27.063099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498d1bc-1478-4789-b51f-40e81d4cf76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:27.064082 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498d1bc-1478-4789-b51f-40e81d4cf76b admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:27.064082 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498d1bc-1478-4789-b51f-40e81d4cf76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:27.064286 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498d1bc-1478-4789-b51f-40e81d4cf76b admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:27.065405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498d1bc-1478-4789-b51f-40e81d4cf76b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:27.067279 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498d1bc-1478-4789-b51f-40e81d4cf76b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:27.078171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1306/2603] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:37:27.082921 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6365a7-5136-4709-aab1-35da3254c43b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:27.095492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6365a7-5136-4709-aab1-35da3254c43b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:27.101758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0358e-e799-4edd-b42a-13186f396ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:27.102534 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0358e-e799-4edd-b42a-13186f396ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:27.102889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0358e-e799-4edd-b42a-13186f396ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:27.103258 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0358e-e799-4edd-b42a-13186f396ffe admin admin] PATH_INFO: `/v3/projects/ebe3c4358dfb4a919b91fd18caff463c/users/8a508d56ad5d43f489e0fc0925eaf704/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:27.105660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea0358e-e799-4edd-b42a-13186f396ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebe3c4358dfb4a919b91fd18caff463c, user_id=8a508d56ad5d43f489e0fc0925eaf704, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:27.108410 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea0358e-e799-4edd-b42a-13186f396ffe admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:27.121669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1298/2604] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:37:27 2022] PUT /identity/v3/projects/ebe3c4358dfb4a919b91fd18caff463c/users/8a508d56ad5d43f489e0fc0925eaf704/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:37:27.131839 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a285b-ed51-486a-8c7b-cc5e3caa9923 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:27.132253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a285b-ed51-486a-8c7b-cc5e3caa9923 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:27.132622 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a285b-ed51-486a-8c7b-cc5e3caa9923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:27.164567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-cb7a285b-ed51-486a-8c7b-cc5e3caa9923 None None] MFA Rules not processed for user `8a508d56ad5d43f489e0fc0925eaf704`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:27.165975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cb7a285b-ed51-486a-8c7b-cc5e3caa9923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:27.261511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1307/2605] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:27.273265 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0358e-e799-4edd-b42a-13186f396ffe admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:27.276332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1ea0358e-e799-4edd-b42a-13186f396ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:27.306912 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:27.352650 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0358e-e799-4edd-b42a-13186f396ffe admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:27.356420 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00877152-fad0-4c72-beaa-8c026bf7c04f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:27.357272 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00877152-fad0-4c72-beaa-8c026bf7c04f service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:27.357639 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00877152-fad0-4c72-beaa-8c026bf7c04f service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:27.358005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00877152-fad0-4c72-beaa-8c026bf7c04f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:27.368440 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00877152-fad0-4c72-beaa-8c026bf7c04f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:27.369993 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00877152-fad0-4c72-beaa-8c026bf7c04f service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:27.380833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1299/2606] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:27.498320 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a285b-ed51-486a-8c7b-cc5e3caa9923 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:27.510198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a285b-ed51-486a-8c7b-cc5e3caa9923 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:27.515113 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c29f87-2603-4e9a-b0c0-f12458edc641 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:27.515952 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c29f87-2603-4e9a-b0c0-f12458edc641 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:27.516397 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c29f87-2603-4e9a-b0c0-f12458edc641 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:27.516823 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c29f87-2603-4e9a-b0c0-f12458edc641 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:27.592896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c29f87-2603-4e9a-b0c0-f12458edc641 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:27.594773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c29f87-2603-4e9a-b0c0-f12458edc641 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:27.608016 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1308/2607] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:27.676711 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357de5a8-0eb4-4f58-82d4-8906766c135d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:27.677199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357de5a8-0eb4-4f58-82d4-8906766c135d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:27.677644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357de5a8-0eb4-4f58-82d4-8906766c135d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:27.710893 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-357de5a8-0eb4-4f58-82d4-8906766c135d None None] MFA Rules not processed for user `72485bc9ed6f4684b77b617597f99e29`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:27.712199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-357de5a8-0eb4-4f58-82d4-8906766c135d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:27.806321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1300/2608] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:27.810310 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c29f87-2603-4e9a-b0c0-f12458edc641 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:27.813064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-60c29f87-2603-4e9a-b0c0-f12458edc641 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:27.849816 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:27.895562 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c29f87-2603-4e9a-b0c0-f12458edc641 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:27.901653 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc9953-c9f3-47ec-ae33-aa7fa73cfe25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:27.902490 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc9953-c9f3-47ec-ae33-aa7fa73cfe25 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:27.902846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc9953-c9f3-47ec-ae33-aa7fa73cfe25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:27.903212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc9953-c9f3-47ec-ae33-aa7fa73cfe25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:27.903963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc9953-c9f3-47ec-ae33-aa7fa73cfe25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:27.906224 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc9953-c9f3-47ec-ae33-aa7fa73cfe25 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:27.937376 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1309/2609] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:37:27 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:37:27.941492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357de5a8-0eb4-4f58-82d4-8906766c135d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:27.952765 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357de5a8-0eb4-4f58-82d4-8906766c135d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:28.011056 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3d074-039c-4468-a184-66f845d0e209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:28.011666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3d074-039c-4468-a184-66f845d0e209 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:28.011942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3d074-039c-4468-a184-66f845d0e209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:28.012286 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3d074-039c-4468-a184-66f845d0e209 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:28.012855 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3d074-039c-4468-a184-66f845d0e209 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:28.015464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3d074-039c-4468-a184-66f845d0e209 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:28.050995 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1301/2610] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:37:27 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:37:28.055015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc9953-c9f3-47ec-ae33-aa7fa73cfe25 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:28.065690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc9953-c9f3-47ec-ae33-aa7fa73cfe25 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:28.070387 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7261b-b49a-4ae0-abd6-c4e55b2a851f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:28.070923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7261b-b49a-4ae0-abd6-c4e55b2a851f admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:28.071205 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7261b-b49a-4ae0-abd6-c4e55b2a851f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:28.071476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7261b-b49a-4ae0-abd6-c4e55b2a851f admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:28.072109 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de7261b-b49a-4ae0-abd6-c4e55b2a851f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:28.073692 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de7261b-b49a-4ae0-abd6-c4e55b2a851f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:28.086646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1310/2611] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:37:28.090584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3d074-039c-4468-a184-66f845d0e209 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:28.101969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3d074-039c-4468-a184-66f845d0e209 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:28.106838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a7083-3638-413b-b607-16c524b7396e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:28.107513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a7083-3638-413b-b607-16c524b7396e admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:28.107795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a7083-3638-413b-b607-16c524b7396e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:28.108073 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a7083-3638-413b-b607-16c524b7396e admin admin] PATH_INFO: `/v3/projects/9bc2e67053b347ea948ecd2b5d497d02/users/aa0cdb6397a94de29ab7434fad1861d9/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:28.124335 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a7083-3638-413b-b607-16c524b7396e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bc2e67053b347ea948ecd2b5d497d02, user_id=aa0cdb6397a94de29ab7434fad1861d9, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:28.127111 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a7083-3638-413b-b607-16c524b7396e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:28.141768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1302/2612] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:37:28 2022] PUT /identity/v3/projects/9bc2e67053b347ea948ecd2b5d497d02/users/aa0cdb6397a94de29ab7434fad1861d9/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:28.145882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7261b-b49a-4ae0-abd6-c4e55b2a851f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:28.163226 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7261b-b49a-4ae0-abd6-c4e55b2a851f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:28.224238 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f136f-3cc9-4fc5-a4af-2111cca954f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:28.225498 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f136f-3cc9-4fc5-a4af-2111cca954f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:28.225773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f136f-3cc9-4fc5-a4af-2111cca954f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:28.226037 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f136f-3cc9-4fc5-a4af-2111cca954f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:28.226707 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f136f-3cc9-4fc5-a4af-2111cca954f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:28.228300 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f136f-3cc9-4fc5-a4af-2111cca954f6 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:28.242550 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1311/2613] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:37:28.243740 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a7083-3638-413b-b607-16c524b7396e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:28.254595 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a7083-3638-413b-b607-16c524b7396e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:28.259804 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b7c11-6dfa-4fb8-aa99-3aa809cb6ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:28.260469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b7c11-6dfa-4fb8-aa99-3aa809cb6ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:28.260744 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b7c11-6dfa-4fb8-aa99-3aa809cb6ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:28.261014 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b7c11-6dfa-4fb8-aa99-3aa809cb6ff4 admin admin] PATH_INFO: `/v3/projects/9bc2e67053b347ea948ecd2b5d497d02/users/aa0cdb6397a94de29ab7434fad1861d9/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:28.262938 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b7c11-6dfa-4fb8-aa99-3aa809cb6ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bc2e67053b347ea948ecd2b5d497d02, user_id=aa0cdb6397a94de29ab7434fad1861d9, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:28.265585 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b7c11-6dfa-4fb8-aa99-3aa809cb6ff4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:28.272816 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:28.277822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1303/2614] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:37:28 2022] PUT /identity/v3/projects/9bc2e67053b347ea948ecd2b5d497d02/users/aa0cdb6397a94de29ab7434fad1861d9/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:32.712914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e4484-5141-49cc-b1e1-8875277fef0a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:32.713395 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e4484-5141-49cc-b1e1-8875277fef0a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:32.713503 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e4484-5141-49cc-b1e1-8875277fef0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:32.741767 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b60e4484-5141-49cc-b1e1-8875277fef0a None None] MFA Rules not processed for user `aa0cdb6397a94de29ab7434fad1861d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:32.743044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b60e4484-5141-49cc-b1e1-8875277fef0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:32.841857 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1312/2615] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:32.863455 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b7c11-6dfa-4fb8-aa99-3aa809cb6ff4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:32.866727 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cd9b7c11-6dfa-4fb8-aa99-3aa809cb6ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:32.959192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b7c11-6dfa-4fb8-aa99-3aa809cb6ff4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:32.965334 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a6fda-48fd-4762-a9f8-f29349979090 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:32.965992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a6fda-48fd-4762-a9f8-f29349979090 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:32.966363 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a6fda-48fd-4762-a9f8-f29349979090 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:32.966709 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a6fda-48fd-4762-a9f8-f29349979090 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:33.043217 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a6fda-48fd-4762-a9f8-f29349979090 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:33.044893 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a6fda-48fd-4762-a9f8-f29349979090 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:33.058859 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1304/2616] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:37.913831 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e4484-5141-49cc-b1e1-8875277fef0a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:37.941523 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:37.987073 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e4484-5141-49cc-b1e1-8875277fef0a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:37.991783 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c4a57-e273-4579-894c-0f782f6f8e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:37.992415 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c4a57-e273-4579-894c-0f782f6f8e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:37.992698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c4a57-e273-4579-894c-0f782f6f8e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:37.992973 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c4a57-e273-4579-894c-0f782f6f8e3b admin admin] PATH_INFO: `/v3/users/79ca99e4bc6b42dabc2cfda9b797a008` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:37.994110 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c4a57-e273-4579-894c-0f782f6f8e3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=79ca99e4bc6b42dabc2cfda9b797a008)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:37.996200 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c4a57-e273-4579-894c-0f782f6f8e3b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:38.051142 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-fc6c4a57-e273-4579-894c-0f782f6f8e3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79ca99e4bc6b42dabc2cfda9b797a008'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:38.059976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-fc6c4a57-e273-4579-894c-0f782f6f8e3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79ca99e4bc6b42dabc2cfda9b797a008'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:38.067928 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-fc6c4a57-e273-4579-894c-0f782f6f8e3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79ca99e4bc6b42dabc2cfda9b797a008'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:38.068643 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-fc6c4a57-e273-4579-894c-0f782f6f8e3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79ca99e4bc6b42dabc2cfda9b797a008'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:38.069302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-fc6c4a57-e273-4579-894c-0f782f6f8e3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79ca99e4bc6b42dabc2cfda9b797a008'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:38.087061 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1313/2617] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:37:37 2022] DELETE /identity/v3/users/79ca99e4bc6b42dabc2cfda9b797a008 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:38.091719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fead0-b3e2-4a79-98bc-f5bf8ccbdb04 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:38.092194 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fead0-b3e2-4a79-98bc-f5bf8ccbdb04 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:38.092386 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fead0-b3e2-4a79-98bc-f5bf8ccbdb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:38.111717 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c75fead0-b3e2-4a79-98bc-f5bf8ccbdb04 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:38.112996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c75fead0-b3e2-4a79-98bc-f5bf8ccbdb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:38.173860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1305/2618] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:38.184535 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c4a57-e273-4579-894c-0f782f6f8e3b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:38.185983 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-fc6c4a57-e273-4579-894c-0f782f6f8e3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:38.244157 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c4a57-e273-4579-894c-0f782f6f8e3b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:38.248289 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd958b-062e-4e95-a112-f1772c10b224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:38.248885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd958b-062e-4e95-a112-f1772c10b224 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:38.249159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd958b-062e-4e95-a112-f1772c10b224 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:38.249425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd958b-062e-4e95-a112-f1772c10b224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:38.258484 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd958b-062e-4e95-a112-f1772c10b224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:38.260474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd958b-062e-4e95-a112-f1772c10b224 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:38.274830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1314/2619] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:38.559188 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75fead0-b3e2-4a79-98bc-f5bf8ccbdb04 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:38.560732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c75fead0-b3e2-4a79-98bc-f5bf8ccbdb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:38.574031 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75fead0-b3e2-4a79-98bc-f5bf8ccbdb04 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:38.578818 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda8be6-7ba4-47d4-a15a-5bf10e7cc6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:38.579450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda8be6-7ba4-47d4-a15a-5bf10e7cc6c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:38.581038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda8be6-7ba4-47d4-a15a-5bf10e7cc6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:38.581038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda8be6-7ba4-47d4-a15a-5bf10e7cc6c1 admin admin] PATH_INFO: `/v3/projects/8602dee834aa40dcbf6fe726494fe7c8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:38.581237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda8be6-7ba4-47d4-a15a-5bf10e7cc6c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8602dee834aa40dcbf6fe726494fe7c8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:38.583100 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda8be6-7ba4-47d4-a15a-5bf10e7cc6c1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:38.621793 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:38.634787 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1dda8be6-7ba4-47d4-a15a-5bf10e7cc6c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8602dee834aa40dcbf6fe726494fe7c8', 'request_id': 'req-1dda8be6-7ba4-47d4-a15a-5bf10e7cc6c1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:38.640384 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1dda8be6-7ba4-47d4-a15a-5bf10e7cc6c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8602dee834aa40dcbf6fe726494fe7c8', 'request_id': 'req-1dda8be6-7ba4-47d4-a15a-5bf10e7cc6c1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:38.647768 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1dda8be6-7ba4-47d4-a15a-5bf10e7cc6c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8602dee834aa40dcbf6fe726494fe7c8', 'request_id': 'req-1dda8be6-7ba4-47d4-a15a-5bf10e7cc6c1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:38.659621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1dda8be6-7ba4-47d4-a15a-5bf10e7cc6c1 admin admin] The token cache is being invalidate because project 8602dee834aa40dcbf6fe726494fe7c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:37:38.659903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1dda8be6-7ba4-47d4-a15a-5bf10e7cc6c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:38.660523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1dda8be6-7ba4-47d4-a15a-5bf10e7cc6c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:38.662659 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1306/2620] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:37:38 2022] DELETE /identity/v3/projects/8602dee834aa40dcbf6fe726494fe7c8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:38.667017 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd958b-062e-4e95-a112-f1772c10b224 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:38.680434 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd958b-062e-4e95-a112-f1772c10b224 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:38.728072 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:38.742565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd61981-98a3-4965-b886-5c39f5dc5d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:38.743314 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd61981-98a3-4965-b886-5c39f5dc5d7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:38.749601 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd61981-98a3-4965-b886-5c39f5dc5d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:38.749939 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd61981-98a3-4965-b886-5c39f5dc5d7d admin admin] PATH_INFO: `/v3/users/8a508d56ad5d43f489e0fc0925eaf704` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:38.751341 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd61981-98a3-4965-b886-5c39f5dc5d7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a508d56ad5d43f489e0fc0925eaf704)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:38.753267 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd61981-98a3-4965-b886-5c39f5dc5d7d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:38.809899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8cd61981-98a3-4965-b886-5c39f5dc5d7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a508d56ad5d43f489e0fc0925eaf704'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:38.817039 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8cd61981-98a3-4965-b886-5c39f5dc5d7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a508d56ad5d43f489e0fc0925eaf704'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:38.826198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8cd61981-98a3-4965-b886-5c39f5dc5d7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a508d56ad5d43f489e0fc0925eaf704'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:38.826868 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8cd61981-98a3-4965-b886-5c39f5dc5d7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a508d56ad5d43f489e0fc0925eaf704'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:38.827477 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8cd61981-98a3-4965-b886-5c39f5dc5d7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a508d56ad5d43f489e0fc0925eaf704'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:38.845325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1315/2621] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:37:38 2022] DELETE /identity/v3/users/8a508d56ad5d43f489e0fc0925eaf704 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:39.123255 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda8be6-7ba4-47d4-a15a-5bf10e7cc6c1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:39.124726 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1dda8be6-7ba4-47d4-a15a-5bf10e7cc6c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:39.139300 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda8be6-7ba4-47d4-a15a-5bf10e7cc6c1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:39.201446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074de5a-e86e-421d-afe9-dfd1aabeb49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:39.202079 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074de5a-e86e-421d-afe9-dfd1aabeb49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:39.202395 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074de5a-e86e-421d-afe9-dfd1aabeb49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:39.202670 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074de5a-e86e-421d-afe9-dfd1aabeb49d admin admin] PATH_INFO: `/v3/projects/ebe3c4358dfb4a919b91fd18caff463c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:39.203788 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074de5a-e86e-421d-afe9-dfd1aabeb49d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebe3c4358dfb4a919b91fd18caff463c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:39.205718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074de5a-e86e-421d-afe9-dfd1aabeb49d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:39.252302 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9074de5a-e86e-421d-afe9-dfd1aabeb49d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebe3c4358dfb4a919b91fd18caff463c', 'request_id': 'req-9074de5a-e86e-421d-afe9-dfd1aabeb49d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:39.258369 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9074de5a-e86e-421d-afe9-dfd1aabeb49d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebe3c4358dfb4a919b91fd18caff463c', 'request_id': 'req-9074de5a-e86e-421d-afe9-dfd1aabeb49d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:39.267645 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9074de5a-e86e-421d-afe9-dfd1aabeb49d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebe3c4358dfb4a919b91fd18caff463c', 'request_id': 'req-9074de5a-e86e-421d-afe9-dfd1aabeb49d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:39.279160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9074de5a-e86e-421d-afe9-dfd1aabeb49d admin admin] The token cache is being invalidate because project ebe3c4358dfb4a919b91fd18caff463c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:37:39.279450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9074de5a-e86e-421d-afe9-dfd1aabeb49d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:39.280087 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9074de5a-e86e-421d-afe9-dfd1aabeb49d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:39.281882 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1307/2622] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:37:39 2022] DELETE /identity/v3/projects/ebe3c4358dfb4a919b91fd18caff463c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:39.286830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494f0a9-a8ef-47c9-b8e0-1511bbf45d94 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:39.287328 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494f0a9-a8ef-47c9-b8e0-1511bbf45d94 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:39.287431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494f0a9-a8ef-47c9-b8e0-1511bbf45d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:39.309179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c494f0a9-a8ef-47c9-b8e0-1511bbf45d94 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:39.310425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c494f0a9-a8ef-47c9-b8e0-1511bbf45d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:39.371001 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1316/2623] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:37:39.379228 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5f884-3158-4038-b4fa-e92c6ba50209 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:39.379511 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5f884-3158-4038-b4fa-e92c6ba50209 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:39.379785 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5f884-3158-4038-b4fa-e92c6ba50209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:39.402116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a9e5f884-3158-4038-b4fa-e92c6ba50209 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:39.403738 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a9e5f884-3158-4038-b4fa-e92c6ba50209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:39.416458 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1308/2624] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:39.417721 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494f0a9-a8ef-47c9-b8e0-1511bbf45d94 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:39.429160 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494f0a9-a8ef-47c9-b8e0-1511bbf45d94 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:39.433729 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df555c53-bbdd-4e5f-952a-0f13791a3c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:39.434408 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df555c53-bbdd-4e5f-952a-0f13791a3c41 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:39.434687 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df555c53-bbdd-4e5f-952a-0f13791a3c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:39.434958 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df555c53-bbdd-4e5f-952a-0f13791a3c41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:39.435639 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df555c53-bbdd-4e5f-952a-0f13791a3c41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:37:39.435964 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df555c53-bbdd-4e5f-952a-0f13791a3c41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:39.437548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df555c53-bbdd-4e5f-952a-0f13791a3c41 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:39.452912 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1317/2625] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:37:39.454901 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5f884-3158-4038-b4fa-e92c6ba50209 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:39.464576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5f884-3158-4038-b4fa-e92c6ba50209 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:39.469236 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02a0a2-fc5e-46cd-aca2-ae8513a1f301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:39.469830 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02a0a2-fc5e-46cd-aca2-ae8513a1f301 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:39.470107 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02a0a2-fc5e-46cd-aca2-ae8513a1f301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:39.470414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02a0a2-fc5e-46cd-aca2-ae8513a1f301 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:39.471039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02a0a2-fc5e-46cd-aca2-ae8513a1f301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:39.473050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02a0a2-fc5e-46cd-aca2-ae8513a1f301 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:39.502111 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1309/2626] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:37:39 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:37:39.506947 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df555c53-bbdd-4e5f-952a-0f13791a3c41 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:39.517506 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df555c53-bbdd-4e5f-952a-0f13791a3c41 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:39.565910 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:39.577410 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b612a-0529-4d01-ad61-c5469f942c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:39.578019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b612a-0529-4d01-ad61-c5469f942c9c admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:39.578324 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b612a-0529-4d01-ad61-c5469f942c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:39.578599 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b612a-0529-4d01-ad61-c5469f942c9c admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:39.579233 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b612a-0529-4d01-ad61-c5469f942c9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:39.581127 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b612a-0529-4d01-ad61-c5469f942c9c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:39.603592 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1318/2627] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:37:39 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:37:39.607727 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02a0a2-fc5e-46cd-aca2-ae8513a1f301 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:39.617838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02a0a2-fc5e-46cd-aca2-ae8513a1f301 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:39.622417 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802ad8d-988e-47cf-8601-6ca9be90b380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:39.622999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802ad8d-988e-47cf-8601-6ca9be90b380 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:39.623271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802ad8d-988e-47cf-8601-6ca9be90b380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:39.623535 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802ad8d-988e-47cf-8601-6ca9be90b380 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:39.624167 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802ad8d-988e-47cf-8601-6ca9be90b380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:39.625787 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802ad8d-988e-47cf-8601-6ca9be90b380 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:39.638479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1310/2628] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:37:39.642346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b612a-0529-4d01-ad61-c5469f942c9c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:39.653896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b612a-0529-4d01-ad61-c5469f942c9c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:39.659117 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73202481-316c-44c3-a791-a6591f770eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:39.659896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73202481-316c-44c3-a791-a6591f770eab admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:39.660210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73202481-316c-44c3-a791-a6591f770eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:39.660487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73202481-316c-44c3-a791-a6591f770eab admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:39.661159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73202481-316c-44c3-a791-a6591f770eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:39.662847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73202481-316c-44c3-a791-a6591f770eab admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:39.679310 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1319/2629] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:37:39.681592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802ad8d-988e-47cf-8601-6ca9be90b380 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:39.693515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802ad8d-988e-47cf-8601-6ca9be90b380 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:39.698592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654d904-937a-4b12-b5d4-48a76920c592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:39.699307 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654d904-937a-4b12-b5d4-48a76920c592 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:39.699585 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654d904-937a-4b12-b5d4-48a76920c592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:39.699852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654d904-937a-4b12-b5d4-48a76920c592 admin admin] PATH_INFO: `/v3/projects/285ee985e0c048afb6a4dc71e85660f4/users/d2d1e347467543e2b33c09551d8926b5/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:39.710772 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:39.719490 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654d904-937a-4b12-b5d4-48a76920c592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=285ee985e0c048afb6a4dc71e85660f4, user_id=d2d1e347467543e2b33c09551d8926b5, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:39.723330 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654d904-937a-4b12-b5d4-48a76920c592 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:39.738638 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1311/2630] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:37:39 2022] PUT /identity/v3/projects/285ee985e0c048afb6a4dc71e85660f4/users/d2d1e347467543e2b33c09551d8926b5/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:39.750474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb80fe-d6d6-4e1a-9d11-b87ded78db8e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:39.750828 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb80fe-d6d6-4e1a-9d11-b87ded78db8e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:39.751039 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb80fe-d6d6-4e1a-9d11-b87ded78db8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:39.782613 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-9dfb80fe-d6d6-4e1a-9d11-b87ded78db8e None None] MFA Rules not processed for user `d2d1e347467543e2b33c09551d8926b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:39.783854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9dfb80fe-d6d6-4e1a-9d11-b87ded78db8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:39.874236 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1320/2631] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:37:39.878479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654d904-937a-4b12-b5d4-48a76920c592 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:39.945376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654d904-937a-4b12-b5d4-48a76920c592 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:39.951234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3200a87-3edd-43c3-b851-8df5ff6349b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:39.951888 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3200a87-3edd-43c3-b851-8df5ff6349b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:39.952144 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3200a87-3edd-43c3-b851-8df5ff6349b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:39.952419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3200a87-3edd-43c3-b851-8df5ff6349b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:39.953092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3200a87-3edd-43c3-b851-8df5ff6349b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:39.955106 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3200a87-3edd-43c3-b851-8df5ff6349b2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:39.985212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1312/2632] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:37:39 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:37:39.990337 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb80fe-d6d6-4e1a-9d11-b87ded78db8e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:40.001407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb80fe-d6d6-4e1a-9d11-b87ded78db8e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:40.060105 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b02c0-b510-4604-992b-708bc4bc8d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:40.060632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b02c0-b510-4604-992b-708bc4bc8d73 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:40.060912 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b02c0-b510-4604-992b-708bc4bc8d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:40.061193 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b02c0-b510-4604-992b-708bc4bc8d73 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:40.061816 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b02c0-b510-4604-992b-708bc4bc8d73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:40.063751 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b02c0-b510-4604-992b-708bc4bc8d73 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:40.089791 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1321/2633] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:37:39 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:37:40.091023 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3200a87-3edd-43c3-b851-8df5ff6349b2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:40.101950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3200a87-3edd-43c3-b851-8df5ff6349b2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:40.106581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4189c-bde0-48f1-9b14-6662916f2fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:40.107171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4189c-bde0-48f1-9b14-6662916f2fcb admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:40.107455 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4189c-bde0-48f1-9b14-6662916f2fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:40.107725 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4189c-bde0-48f1-9b14-6662916f2fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:40.108356 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4189c-bde0-48f1-9b14-6662916f2fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:40.109973 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4189c-bde0-48f1-9b14-6662916f2fcb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:40.121877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1313/2634] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:37:40.126045 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b02c0-b510-4604-992b-708bc4bc8d73 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:40.137829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b02c0-b510-4604-992b-708bc4bc8d73 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:40.142553 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c0bc5-3410-4790-90b2-4939c21808c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:40.143201 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c0bc5-3410-4790-90b2-4939c21808c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:40.143480 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c0bc5-3410-4790-90b2-4939c21808c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:40.143755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c0bc5-3410-4790-90b2-4939c21808c7 admin admin] PATH_INFO: `/v3/projects/27a60a4198c743d0a72511b795b61864/users/b213922b84d7461b9fd5bff5229af73b/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:40.158947 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c0bc5-3410-4790-90b2-4939c21808c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27a60a4198c743d0a72511b795b61864, user_id=b213922b84d7461b9fd5bff5229af73b, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:40.162190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c0bc5-3410-4790-90b2-4939c21808c7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:40.175862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1322/2635] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:37:40 2022] PUT /identity/v3/projects/27a60a4198c743d0a72511b795b61864/users/b213922b84d7461b9fd5bff5229af73b/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:40.179742 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4189c-bde0-48f1-9b14-6662916f2fcb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:40.190363 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4189c-bde0-48f1-9b14-6662916f2fcb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:40.248037 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b29990-97ff-4034-9cdf-434f7f7f245c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:40.248640 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b29990-97ff-4034-9cdf-434f7f7f245c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:40.248919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b29990-97ff-4034-9cdf-434f7f7f245c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:40.249195 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b29990-97ff-4034-9cdf-434f7f7f245c admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:40.249827 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b29990-97ff-4034-9cdf-434f7f7f245c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:40.251481 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b29990-97ff-4034-9cdf-434f7f7f245c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:40.263418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1314/2636] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:37:40.267591 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c0bc5-3410-4790-90b2-4939c21808c7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:40.278461 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c0bc5-3410-4790-90b2-4939c21808c7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:40.283064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31aeaf-313c-434a-89fb-77a71148c341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:40.283698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31aeaf-313c-434a-89fb-77a71148c341 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:40.283976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31aeaf-313c-434a-89fb-77a71148c341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:40.284254 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31aeaf-313c-434a-89fb-77a71148c341 admin admin] PATH_INFO: `/v3/projects/27a60a4198c743d0a72511b795b61864/users/b213922b84d7461b9fd5bff5229af73b/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:40.286169 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd31aeaf-313c-434a-89fb-77a71148c341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27a60a4198c743d0a72511b795b61864, user_id=b213922b84d7461b9fd5bff5229af73b, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:40.288810 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd31aeaf-313c-434a-89fb-77a71148c341 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:40.299689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1323/2637] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:37:40 2022] PUT /identity/v3/projects/27a60a4198c743d0a72511b795b61864/users/b213922b84d7461b9fd5bff5229af73b/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:40.309941 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342f03e-1671-4b08-9e78-63d25f768c98 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:40.310316 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342f03e-1671-4b08-9e78-63d25f768c98 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:40.310531 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342f03e-1671-4b08-9e78-63d25f768c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:40.339469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e342f03e-1671-4b08-9e78-63d25f768c98 None None] MFA Rules not processed for user `b213922b84d7461b9fd5bff5229af73b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:40.340706 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e342f03e-1671-4b08-9e78-63d25f768c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:40.387370 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:40.419573 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1315/2638] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:40 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:40.430212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31aeaf-313c-434a-89fb-77a71148c341 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:40.431931 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-fd31aeaf-313c-434a-89fb-77a71148c341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:40.448590 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:40.493713 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31aeaf-313c-434a-89fb-77a71148c341 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:40.497383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d68747-43cb-4b39-bc86-f8fcb4d8e019 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:40.497987 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d68747-43cb-4b39-bc86-f8fcb4d8e019 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:40.498271 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d68747-43cb-4b39-bc86-f8fcb4d8e019 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:40.498557 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d68747-43cb-4b39-bc86-f8fcb4d8e019 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:40.508357 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d68747-43cb-4b39-bc86-f8fcb4d8e019 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:40.509760 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d68747-43cb-4b39-bc86-f8fcb4d8e019 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:40.519308 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1324/2639] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:40 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:40.547564 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342f03e-1671-4b08-9e78-63d25f768c98 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:40.610889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342f03e-1671-4b08-9e78-63d25f768c98 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:40.615883 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d50d3-0cd0-45a7-9d44-e81403250e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:40.616507 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d50d3-0cd0-45a7-9d44-e81403250e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:40.616785 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d50d3-0cd0-45a7-9d44-e81403250e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:40.617059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d50d3-0cd0-45a7-9d44-e81403250e57 admin admin] PATH_INFO: `/v3/users/d2d1e347467543e2b33c09551d8926b5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:40.618196 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d50d3-0cd0-45a7-9d44-e81403250e57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2d1e347467543e2b33c09551d8926b5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:40.620148 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d50d3-0cd0-45a7-9d44-e81403250e57 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:40.667283 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c53d50d3-0cd0-45a7-9d44-e81403250e57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2d1e347467543e2b33c09551d8926b5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:40.677587 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c53d50d3-0cd0-45a7-9d44-e81403250e57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2d1e347467543e2b33c09551d8926b5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:40.684991 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c53d50d3-0cd0-45a7-9d44-e81403250e57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2d1e347467543e2b33c09551d8926b5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:40.685630 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c53d50d3-0cd0-45a7-9d44-e81403250e57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2d1e347467543e2b33c09551d8926b5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:40.686265 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c53d50d3-0cd0-45a7-9d44-e81403250e57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2d1e347467543e2b33c09551d8926b5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:40.704729 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1316/2640] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:37:40 2022] DELETE /identity/v3/users/d2d1e347467543e2b33c09551d8926b5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:40.709602 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd814ef6-5ace-490c-989a-c3ac31e538f9 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:40.709950 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd814ef6-5ace-490c-989a-c3ac31e538f9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:40.710232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd814ef6-5ace-490c-989a-c3ac31e538f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:40.728894 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-dd814ef6-5ace-490c-989a-c3ac31e538f9 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:40.730118 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dd814ef6-5ace-490c-989a-c3ac31e538f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:40.790770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1325/2641] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:40.801019 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d50d3-0cd0-45a7-9d44-e81403250e57 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:40.803078 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c53d50d3-0cd0-45a7-9d44-e81403250e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:40.862156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d50d3-0cd0-45a7-9d44-e81403250e57 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:40.865720 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e384f4-cd04-414d-a807-48993d7602cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:40.866409 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e384f4-cd04-414d-a807-48993d7602cb service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:40.866692 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e384f4-cd04-414d-a807-48993d7602cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:40.866961 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e384f4-cd04-414d-a807-48993d7602cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:40.876172 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e384f4-cd04-414d-a807-48993d7602cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:40.877604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e384f4-cd04-414d-a807-48993d7602cb service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:40.887073 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1317/2642] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:41.168386 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd814ef6-5ace-490c-989a-c3ac31e538f9 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:41.169901 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dd814ef6-5ace-490c-989a-c3ac31e538f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:41.183177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd814ef6-5ace-490c-989a-c3ac31e538f9 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:41.187738 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f9f4b-8e5c-4354-a74b-9056013b828d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:41.188371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f9f4b-8e5c-4354-a74b-9056013b828d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:41.188998 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f9f4b-8e5c-4354-a74b-9056013b828d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:41.189253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f9f4b-8e5c-4354-a74b-9056013b828d admin admin] PATH_INFO: `/v3/projects/285ee985e0c048afb6a4dc71e85660f4` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:41.190807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f9f4b-8e5c-4354-a74b-9056013b828d admin admin] RBAC: Authorizing `identity:delete_project(project_id=285ee985e0c048afb6a4dc71e85660f4)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:41.199684 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f9f4b-8e5c-4354-a74b-9056013b828d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:41.249542 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0f6f9f4b-8e5c-4354-a74b-9056013b828d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '285ee985e0c048afb6a4dc71e85660f4', 'request_id': 'req-0f6f9f4b-8e5c-4354-a74b-9056013b828d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:41.253655 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0f6f9f4b-8e5c-4354-a74b-9056013b828d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '285ee985e0c048afb6a4dc71e85660f4', 'request_id': 'req-0f6f9f4b-8e5c-4354-a74b-9056013b828d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:41.261372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0f6f9f4b-8e5c-4354-a74b-9056013b828d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '285ee985e0c048afb6a4dc71e85660f4', 'request_id': 'req-0f6f9f4b-8e5c-4354-a74b-9056013b828d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:41.272677 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0f6f9f4b-8e5c-4354-a74b-9056013b828d admin admin] The token cache is being invalidate because project 285ee985e0c048afb6a4dc71e85660f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:37:41.272980 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0f6f9f4b-8e5c-4354-a74b-9056013b828d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:41.273608 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0f6f9f4b-8e5c-4354-a74b-9056013b828d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:41.275137 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1326/2643] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:37:41 2022] DELETE /identity/v3/projects/285ee985e0c048afb6a4dc71e85660f4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:41.279077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e384f4-cd04-414d-a807-48993d7602cb service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:41.286554 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:41.291638 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e384f4-cd04-414d-a807-48993d7602cb service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:41.352466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a71d3-28cd-46c5-be65-a37afe57edda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:41.353088 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a71d3-28cd-46c5-be65-a37afe57edda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:41.353364 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a71d3-28cd-46c5-be65-a37afe57edda admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:41.353632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a71d3-28cd-46c5-be65-a37afe57edda admin admin] PATH_INFO: `/v3/users/b213922b84d7461b9fd5bff5229af73b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:41.354775 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a71d3-28cd-46c5-be65-a37afe57edda admin admin] RBAC: Authorizing `identity:delete_user(user_id=b213922b84d7461b9fd5bff5229af73b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:41.356723 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a71d3-28cd-46c5-be65-a37afe57edda admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:41.402197 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-243a71d3-28cd-46c5-be65-a37afe57edda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b213922b84d7461b9fd5bff5229af73b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:41.411024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-243a71d3-28cd-46c5-be65-a37afe57edda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b213922b84d7461b9fd5bff5229af73b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:41.422005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-243a71d3-28cd-46c5-be65-a37afe57edda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b213922b84d7461b9fd5bff5229af73b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:41.422753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-243a71d3-28cd-46c5-be65-a37afe57edda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b213922b84d7461b9fd5bff5229af73b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:41.423400 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-243a71d3-28cd-46c5-be65-a37afe57edda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b213922b84d7461b9fd5bff5229af73b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:41.443993 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1318/2644] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:37:41 2022] DELETE /identity/v3/users/b213922b84d7461b9fd5bff5229af73b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:41.718973 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f9f4b-8e5c-4354-a74b-9056013b828d admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:41.720412 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0f6f9f4b-8e5c-4354-a74b-9056013b828d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:41.737822 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f9f4b-8e5c-4354-a74b-9056013b828d admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:41.779426 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:41.801489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076d2cb-ec68-4667-9e92-b0cf29a1181c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:41.802117 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076d2cb-ec68-4667-9e92-b0cf29a1181c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:41.802457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076d2cb-ec68-4667-9e92-b0cf29a1181c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:41.802722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076d2cb-ec68-4667-9e92-b0cf29a1181c admin admin] PATH_INFO: `/v3/projects/27a60a4198c743d0a72511b795b61864` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:41.803922 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076d2cb-ec68-4667-9e92-b0cf29a1181c admin admin] RBAC: Authorizing `identity:delete_project(project_id=27a60a4198c743d0a72511b795b61864)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:41.805819 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076d2cb-ec68-4667-9e92-b0cf29a1181c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:41.852860 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3076d2cb-ec68-4667-9e92-b0cf29a1181c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27a60a4198c743d0a72511b795b61864', 'request_id': 'req-3076d2cb-ec68-4667-9e92-b0cf29a1181c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:41.858463 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3076d2cb-ec68-4667-9e92-b0cf29a1181c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27a60a4198c743d0a72511b795b61864', 'request_id': 'req-3076d2cb-ec68-4667-9e92-b0cf29a1181c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:41.866348 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3076d2cb-ec68-4667-9e92-b0cf29a1181c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27a60a4198c743d0a72511b795b61864', 'request_id': 'req-3076d2cb-ec68-4667-9e92-b0cf29a1181c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:41.877596 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3076d2cb-ec68-4667-9e92-b0cf29a1181c admin admin] The token cache is being invalidate because project 27a60a4198c743d0a72511b795b61864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:37:41.877900 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3076d2cb-ec68-4667-9e92-b0cf29a1181c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:41.878558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3076d2cb-ec68-4667-9e92-b0cf29a1181c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:41.880048 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1327/2645] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:37:41 2022] DELETE /identity/v3/projects/27a60a4198c743d0a72511b795b61864 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:41.885864 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc1e33-4a53-4481-af21-cabb63a88a5f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:41.886149 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc1e33-4a53-4481-af21-cabb63a88a5f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:41.886459 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc1e33-4a53-4481-af21-cabb63a88a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:41.908227 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bfdc1e33-4a53-4481-af21-cabb63a88a5f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:41.909436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bfdc1e33-4a53-4481-af21-cabb63a88a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:41.977163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1319/2646] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:41.985534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1a325-8523-46a4-9829-cb2c16273f7e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:41.985815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1a325-8523-46a4-9829-cb2c16273f7e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:41.986087 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1a325-8523-46a4-9829-cb2c16273f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:42.005253 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-28f1a325-8523-46a4-9829-cb2c16273f7e None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:42.006484 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-28f1a325-8523-46a4-9829-cb2c16273f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:42.014748 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1328/2647] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:42.019513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc1e33-4a53-4481-af21-cabb63a88a5f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:42.030062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc1e33-4a53-4481-af21-cabb63a88a5f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:42.034926 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42be11-bf7e-4c9b-94ce-6d9b74c8f1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:42.035521 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42be11-bf7e-4c9b-94ce-6d9b74c8f1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:42.035793 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42be11-bf7e-4c9b-94ce-6d9b74c8f1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:42.036058 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42be11-bf7e-4c9b-94ce-6d9b74c8f1d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:42.036750 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42be11-bf7e-4c9b-94ce-6d9b74c8f1d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:37:42.037051 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42be11-bf7e-4c9b-94ce-6d9b74c8f1d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:42.038719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42be11-bf7e-4c9b-94ce-6d9b74c8f1d8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:42.049998 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1320/2648] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:37:42.054783 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1a325-8523-46a4-9829-cb2c16273f7e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:42.064405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1a325-8523-46a4-9829-cb2c16273f7e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:42.069803 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98e704-1735-4b35-8d72-fcef17d557db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:42.070461 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98e704-1735-4b35-8d72-fcef17d557db admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:42.070737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98e704-1735-4b35-8d72-fcef17d557db admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:42.071003 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98e704-1735-4b35-8d72-fcef17d557db admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:42.071630 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98e704-1735-4b35-8d72-fcef17d557db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:42.073501 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98e704-1735-4b35-8d72-fcef17d557db admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:42.102092 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1329/2649] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:37:42 2022] POST /identity/v3/projects => generated 395 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:37:42.106644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42be11-bf7e-4c9b-94ce-6d9b74c8f1d8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:42.117272 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42be11-bf7e-4c9b-94ce-6d9b74c8f1d8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:42.124872 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:42.175900 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274072aa-fd48-4f05-bee7-1140bdb1ef92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:42.176418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274072aa-fd48-4f05-bee7-1140bdb1ef92 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:42.176698 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274072aa-fd48-4f05-bee7-1140bdb1ef92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:42.176983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274072aa-fd48-4f05-bee7-1140bdb1ef92 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:42.177596 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274072aa-fd48-4f05-bee7-1140bdb1ef92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:42.180065 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274072aa-fd48-4f05-bee7-1140bdb1ef92 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:42.201925 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1321/2650] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:37:42 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:37:42.206207 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98e704-1735-4b35-8d72-fcef17d557db admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:42.217632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98e704-1735-4b35-8d72-fcef17d557db admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:42.223051 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1748afd-8906-448b-b98e-41c18d38b6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:42.223644 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1748afd-8906-448b-b98e-41c18d38b6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:42.223917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1748afd-8906-448b-b98e-41c18d38b6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:42.224187 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1748afd-8906-448b-b98e-41c18d38b6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:42.224827 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1748afd-8906-448b-b98e-41c18d38b6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:42.227332 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1748afd-8906-448b-b98e-41c18d38b6f8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:42.238667 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1330/2651] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:37:42.243313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274072aa-fd48-4f05-bee7-1140bdb1ef92 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:42.253296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274072aa-fd48-4f05-bee7-1140bdb1ef92 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:42.257856 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680cbb59-8a4a-4046-85cb-95e4c464fa20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:42.258463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cbb59-8a4a-4046-85cb-95e4c464fa20 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:42.258739 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cbb59-8a4a-4046-85cb-95e4c464fa20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:42.259006 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cbb59-8a4a-4046-85cb-95e4c464fa20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:42.259636 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680cbb59-8a4a-4046-85cb-95e4c464fa20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:42.261281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680cbb59-8a4a-4046-85cb-95e4c464fa20 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:42.273998 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1322/2652] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:37:42.278273 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1748afd-8906-448b-b98e-41c18d38b6f8 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:42.288888 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1748afd-8906-448b-b98e-41c18d38b6f8 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:42.293467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254516a-804d-4cba-ae34-f4004aaaff40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:42.294151 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254516a-804d-4cba-ae34-f4004aaaff40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:42.294451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254516a-804d-4cba-ae34-f4004aaaff40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:42.294781 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254516a-804d-4cba-ae34-f4004aaaff40 admin admin] PATH_INFO: `/v3/projects/4ecdd5d0b15b4a828b1e226dac50e628/users/d4f507a1c017410891ddb2438c0fd334/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:42.309217 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254516a-804d-4cba-ae34-f4004aaaff40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ecdd5d0b15b4a828b1e226dac50e628, user_id=d4f507a1c017410891ddb2438c0fd334, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:42.312043 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254516a-804d-4cba-ae34-f4004aaaff40 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:42.325239 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1331/2653] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:37:42 2022] PUT /identity/v3/projects/4ecdd5d0b15b4a828b1e226dac50e628/users/d4f507a1c017410891ddb2438c0fd334/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:37:42.336921 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9182b7-da30-4606-a0e4-15f7d7d9f702 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:42.337405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9182b7-da30-4606-a0e4-15f7d7d9f702 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:42.337651 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9182b7-da30-4606-a0e4-15f7d7d9f702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:42.493087 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ab9182b7-da30-4606-a0e4-15f7d7d9f702 None None] MFA Rules not processed for user `d4f507a1c017410891ddb2438c0fd334`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:42.494340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ab9182b7-da30-4606-a0e4-15f7d7d9f702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:42.559625 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1323/2654] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:42 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:42.564279 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254516a-804d-4cba-ae34-f4004aaaff40 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:42.626221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254516a-804d-4cba-ae34-f4004aaaff40 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:42.630949 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec60d9-c958-46d9-8471-285f452a6d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:42.631549 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec60d9-c958-46d9-8471-285f452a6d2f admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:42.631827 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec60d9-c958-46d9-8471-285f452a6d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:42.632099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec60d9-c958-46d9-8471-285f452a6d2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:42.632724 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec60d9-c958-46d9-8471-285f452a6d2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:42.634648 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec60d9-c958-46d9-8471-285f452a6d2f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:42.645659 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:42.665256 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1332/2655] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:37:42 2022] POST /identity/v3/projects => generated 401 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:37:42.668694 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9182b7-da30-4606-a0e4-15f7d7d9f702 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:42.679392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9182b7-da30-4606-a0e4-15f7d7d9f702 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:42.735669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c4302-ff1f-4c86-8820-ba94a80ae188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:42.736168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c4302-ff1f-4c86-8820-ba94a80ae188 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:42.736447 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c4302-ff1f-4c86-8820-ba94a80ae188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:42.736714 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c4302-ff1f-4c86-8820-ba94a80ae188 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:42.737398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c4302-ff1f-4c86-8820-ba94a80ae188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:42.739358 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c4302-ff1f-4c86-8820-ba94a80ae188 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:42.764126 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1324/2656] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:37:42 2022] POST /identity/v3/users => generated 307 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:37:42.765373 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec60d9-c958-46d9-8471-285f452a6d2f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:42.776316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec60d9-c958-46d9-8471-285f452a6d2f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:42.781036 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810bd1d-7772-462e-8022-a5ab6fc89802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:42.781667 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810bd1d-7772-462e-8022-a5ab6fc89802 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:42.781907 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810bd1d-7772-462e-8022-a5ab6fc89802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:42.782176 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810bd1d-7772-462e-8022-a5ab6fc89802 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:42.782852 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0810bd1d-7772-462e-8022-a5ab6fc89802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:42.784472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0810bd1d-7772-462e-8022-a5ab6fc89802 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:42.795963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1333/2657] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:37:42.800128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c4302-ff1f-4c86-8820-ba94a80ae188 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:42.810171 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c4302-ff1f-4c86-8820-ba94a80ae188 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:42.815001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18544e77-e1f7-402d-99bb-7922af1dcad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:42.815894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18544e77-e1f7-402d-99bb-7922af1dcad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:42.816168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18544e77-e1f7-402d-99bb-7922af1dcad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:42.816461 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18544e77-e1f7-402d-99bb-7922af1dcad4 admin admin] PATH_INFO: `/v3/projects/f1035cd2d46a4ac8995f2b8ccd1497c6/users/ebb02ace55a240788b0402331ee31a62/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:42.819485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:42.833247 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18544e77-e1f7-402d-99bb-7922af1dcad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1035cd2d46a4ac8995f2b8ccd1497c6, user_id=ebb02ace55a240788b0402331ee31a62, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:42.836493 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18544e77-e1f7-402d-99bb-7922af1dcad4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:42.849920 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1325/2658] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:37:42 2022] PUT /identity/v3/projects/f1035cd2d46a4ac8995f2b8ccd1497c6/users/ebb02ace55a240788b0402331ee31a62/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:42.853850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810bd1d-7772-462e-8022-a5ab6fc89802 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:42.863946 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810bd1d-7772-462e-8022-a5ab6fc89802 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:42.921003 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875084af-2b25-49d5-9d64-e0b080aee381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:42.921608 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875084af-2b25-49d5-9d64-e0b080aee381 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:42.921885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875084af-2b25-49d5-9d64-e0b080aee381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:42.922156 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875084af-2b25-49d5-9d64-e0b080aee381 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:42.922824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875084af-2b25-49d5-9d64-e0b080aee381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:42.924412 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875084af-2b25-49d5-9d64-e0b080aee381 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:42.934013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1334/2659] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:37:42.938366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18544e77-e1f7-402d-99bb-7922af1dcad4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:42.949467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18544e77-e1f7-402d-99bb-7922af1dcad4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:42.954072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc38ad-25f5-411f-b3df-6c86ef14a433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:42.954748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc38ad-25f5-411f-b3df-6c86ef14a433 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:42.955030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc38ad-25f5-411f-b3df-6c86ef14a433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:42.955301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc38ad-25f5-411f-b3df-6c86ef14a433 admin admin] PATH_INFO: `/v3/projects/f1035cd2d46a4ac8995f2b8ccd1497c6/users/ebb02ace55a240788b0402331ee31a62/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:42.957246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc38ad-25f5-411f-b3df-6c86ef14a433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1035cd2d46a4ac8995f2b8ccd1497c6, user_id=ebb02ace55a240788b0402331ee31a62, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:42.960038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc38ad-25f5-411f-b3df-6c86ef14a433 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:42.972000 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1326/2660] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:37:42 2022] PUT /identity/v3/projects/f1035cd2d46a4ac8995f2b8ccd1497c6/users/ebb02ace55a240788b0402331ee31a62/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:42.982228 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a4955-f4bc-4443-83c2-91d38ed69151 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:42.982566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a4955-f4bc-4443-83c2-91d38ed69151 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:42.982889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a4955-f4bc-4443-83c2-91d38ed69151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:43.014035 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6f3a4955-f4bc-4443-83c2-91d38ed69151 None None] MFA Rules not processed for user `ebb02ace55a240788b0402331ee31a62`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:43.015378 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6f3a4955-f4bc-4443-83c2-91d38ed69151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:43.091085 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1335/2661] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:42 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:43.101578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc38ad-25f5-411f-b3df-6c86ef14a433 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:43.103277 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-03dc38ad-25f5-411f-b3df-6c86ef14a433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:43.160897 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc38ad-25f5-411f-b3df-6c86ef14a433 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:43.164573 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344f470-f9ce-4db3-a5e4-1801f7d4ce3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:43.165170 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344f470-f9ce-4db3-a5e4-1801f7d4ce3e service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:43.165449 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344f470-f9ce-4db3-a5e4-1801f7d4ce3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:43.165719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344f470-f9ce-4db3-a5e4-1801f7d4ce3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:43.174569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344f470-f9ce-4db3-a5e4-1801f7d4ce3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:43.175995 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344f470-f9ce-4db3-a5e4-1801f7d4ce3e service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:43.185875 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1327/2662] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:43 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:43.263236 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a4955-f4bc-4443-83c2-91d38ed69151 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:43.273208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a4955-f4bc-4443-83c2-91d38ed69151 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:43.276910 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225c975-9c2a-49e9-93de-199d89a36b22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:43.277521 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225c975-9c2a-49e9-93de-199d89a36b22 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:43.277801 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225c975-9c2a-49e9-93de-199d89a36b22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:43.278075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225c975-9c2a-49e9-93de-199d89a36b22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:43.336015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225c975-9c2a-49e9-93de-199d89a36b22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:43.337445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225c975-9c2a-49e9-93de-199d89a36b22 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:43.342019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:43.347552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1336/2663] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:43 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:48.461435 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344f470-f9ce-4db3-a5e4-1801f7d4ce3e service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:48.524019 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344f470-f9ce-4db3-a5e4-1801f7d4ce3e service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:48.529066 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f97ef-831d-4334-b1d1-5e451dfe3808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:48.529704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f97ef-831d-4334-b1d1-5e451dfe3808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:48.529983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f97ef-831d-4334-b1d1-5e451dfe3808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:48.530282 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f97ef-831d-4334-b1d1-5e451dfe3808 admin admin] PATH_INFO: `/v3/users/d4f507a1c017410891ddb2438c0fd334` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:48.531409 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f97ef-831d-4334-b1d1-5e451dfe3808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4f507a1c017410891ddb2438c0fd334)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:48.533332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f97ef-831d-4334-b1d1-5e451dfe3808 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:48.582126 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d26f97ef-831d-4334-b1d1-5e451dfe3808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4f507a1c017410891ddb2438c0fd334'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:48.591245 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d26f97ef-831d-4334-b1d1-5e451dfe3808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4f507a1c017410891ddb2438c0fd334'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:48.599785 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d26f97ef-831d-4334-b1d1-5e451dfe3808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4f507a1c017410891ddb2438c0fd334'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:48.600457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d26f97ef-831d-4334-b1d1-5e451dfe3808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4f507a1c017410891ddb2438c0fd334'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:48.601076 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d26f97ef-831d-4334-b1d1-5e451dfe3808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4f507a1c017410891ddb2438c0fd334'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:48.611667 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:48.619944 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1328/2664] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:37:48 2022] DELETE /identity/v3/users/d4f507a1c017410891ddb2438c0fd334 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:48.624361 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2437b-76ed-40c9-beb7-262a5865c582 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:48.624885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2437b-76ed-40c9-beb7-262a5865c582 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:48.624987 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2437b-76ed-40c9-beb7-262a5865c582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:48.643370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-0aa2437b-76ed-40c9-beb7-262a5865c582 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:48.644574 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0aa2437b-76ed-40c9-beb7-262a5865c582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:48.702155 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1337/2665] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:37:48.712604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f97ef-831d-4334-b1d1-5e451dfe3808 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:48.713990 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d26f97ef-831d-4334-b1d1-5e451dfe3808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:48.769820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f97ef-831d-4334-b1d1-5e451dfe3808 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:48.773533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbca99-a146-4873-9dfb-ce5a6429a494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:48.774191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbca99-a146-4873-9dfb-ce5a6429a494 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:48.774937 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbca99-a146-4873-9dfb-ce5a6429a494 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:48.775207 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbca99-a146-4873-9dfb-ce5a6429a494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:48.784237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbca99-a146-4873-9dfb-ce5a6429a494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:48.785652 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbca99-a146-4873-9dfb-ce5a6429a494 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:48.795607 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1329/2666] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:49.073897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2437b-76ed-40c9-beb7-262a5865c582 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:49.075449 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0aa2437b-76ed-40c9-beb7-262a5865c582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:49.090823 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2437b-76ed-40c9-beb7-262a5865c582 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:49.096241 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d45b8a-e344-4434-bb47-bc470b92bf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:49.096875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d45b8a-e344-4434-bb47-bc470b92bf04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:49.097160 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d45b8a-e344-4434-bb47-bc470b92bf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:49.097751 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d45b8a-e344-4434-bb47-bc470b92bf04 admin admin] PATH_INFO: `/v3/projects/4ecdd5d0b15b4a828b1e226dac50e628` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:49.098873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d45b8a-e344-4434-bb47-bc470b92bf04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ecdd5d0b15b4a828b1e226dac50e628)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:49.102088 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d45b8a-e344-4434-bb47-bc470b92bf04 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:49.148695 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-73d45b8a-e344-4434-bb47-bc470b92bf04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ecdd5d0b15b4a828b1e226dac50e628', 'request_id': 'req-73d45b8a-e344-4434-bb47-bc470b92bf04'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:49.155060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-73d45b8a-e344-4434-bb47-bc470b92bf04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ecdd5d0b15b4a828b1e226dac50e628', 'request_id': 'req-73d45b8a-e344-4434-bb47-bc470b92bf04'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:49.162553 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-73d45b8a-e344-4434-bb47-bc470b92bf04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ecdd5d0b15b4a828b1e226dac50e628', 'request_id': 'req-73d45b8a-e344-4434-bb47-bc470b92bf04'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:49.173675 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-73d45b8a-e344-4434-bb47-bc470b92bf04 admin admin] The token cache is being invalidate because project 4ecdd5d0b15b4a828b1e226dac50e628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:37:49.173961 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-73d45b8a-e344-4434-bb47-bc470b92bf04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:49.174763 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-73d45b8a-e344-4434-bb47-bc470b92bf04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:49.177254 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1338/2667] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:37:49 2022] DELETE /identity/v3/projects/4ecdd5d0b15b4a828b1e226dac50e628 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:49.182491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbca99-a146-4873-9dfb-ce5a6429a494 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:49.192808 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbca99-a146-4873-9dfb-ce5a6429a494 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:49.248249 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685687d-7b3d-4bc5-a7dd-a74473a8e710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:49.248871 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685687d-7b3d-4bc5-a7dd-a74473a8e710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:49.249150 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685687d-7b3d-4bc5-a7dd-a74473a8e710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:49.249449 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685687d-7b3d-4bc5-a7dd-a74473a8e710 admin admin] PATH_INFO: `/v3/users/ebb02ace55a240788b0402331ee31a62` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:49.250629 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685687d-7b3d-4bc5-a7dd-a74473a8e710 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebb02ace55a240788b0402331ee31a62)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:49.252533 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685687d-7b3d-4bc5-a7dd-a74473a8e710 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:49.295892 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b685687d-7b3d-4bc5-a7dd-a74473a8e710 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebb02ace55a240788b0402331ee31a62'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:49.303365 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b685687d-7b3d-4bc5-a7dd-a74473a8e710 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebb02ace55a240788b0402331ee31a62'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:49.312836 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b685687d-7b3d-4bc5-a7dd-a74473a8e710 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebb02ace55a240788b0402331ee31a62'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:49.313484 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b685687d-7b3d-4bc5-a7dd-a74473a8e710 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebb02ace55a240788b0402331ee31a62'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:49.314161 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b685687d-7b3d-4bc5-a7dd-a74473a8e710 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebb02ace55a240788b0402331ee31a62'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:49.332396 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1330/2668] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:37:49 2022] DELETE /identity/v3/users/ebb02ace55a240788b0402331ee31a62 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:49.615349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d45b8a-e344-4434-bb47-bc470b92bf04 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:49.616926 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-73d45b8a-e344-4434-bb47-bc470b92bf04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:49.632333 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d45b8a-e344-4434-bb47-bc470b92bf04 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:49.690447 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39858c9a-6e07-4915-87f2-c699d5f9dfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:49.691073 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39858c9a-6e07-4915-87f2-c699d5f9dfea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:49.691353 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39858c9a-6e07-4915-87f2-c699d5f9dfea admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:49.691626 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39858c9a-6e07-4915-87f2-c699d5f9dfea admin admin] PATH_INFO: `/v3/projects/f1035cd2d46a4ac8995f2b8ccd1497c6` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:49.693017 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39858c9a-6e07-4915-87f2-c699d5f9dfea admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1035cd2d46a4ac8995f2b8ccd1497c6)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:49.694956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39858c9a-6e07-4915-87f2-c699d5f9dfea admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:49.726972 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:49.747340 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-39858c9a-6e07-4915-87f2-c699d5f9dfea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1035cd2d46a4ac8995f2b8ccd1497c6', 'request_id': 'req-39858c9a-6e07-4915-87f2-c699d5f9dfea'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:49.753507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-39858c9a-6e07-4915-87f2-c699d5f9dfea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1035cd2d46a4ac8995f2b8ccd1497c6', 'request_id': 'req-39858c9a-6e07-4915-87f2-c699d5f9dfea'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:49.761552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-39858c9a-6e07-4915-87f2-c699d5f9dfea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1035cd2d46a4ac8995f2b8ccd1497c6', 'request_id': 'req-39858c9a-6e07-4915-87f2-c699d5f9dfea'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:49.773022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-39858c9a-6e07-4915-87f2-c699d5f9dfea admin admin] The token cache is being invalidate because project f1035cd2d46a4ac8995f2b8ccd1497c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:37:49.773316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-39858c9a-6e07-4915-87f2-c699d5f9dfea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:49.773978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-39858c9a-6e07-4915-87f2-c699d5f9dfea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:37:49.776118 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1339/2669] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:37:49 2022] DELETE /identity/v3/projects/f1035cd2d46a4ac8995f2b8ccd1497c6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:37:49.780956 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c670a2-50c3-4bce-b762-36d075f0ce75 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:49.781486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c670a2-50c3-4bce-b762-36d075f0ce75 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:49.781486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c670a2-50c3-4bce-b762-36d075f0ce75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:49.801688 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-44c670a2-50c3-4bce-b762-36d075f0ce75 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:49.802947 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-44c670a2-50c3-4bce-b762-36d075f0ce75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:49.813655 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:49.866611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1331/2670] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:49.875546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0104ca4-c59d-4d79-a900-9aa92baff3f1 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:49.875832 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0104ca4-c59d-4d79-a900-9aa92baff3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:49.876106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0104ca4-c59d-4d79-a900-9aa92baff3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:49.896167 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f0104ca4-c59d-4d79-a900-9aa92baff3f1 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:49.897378 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f0104ca4-c59d-4d79-a900-9aa92baff3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:49.905969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1340/2671] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:49.910098 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c670a2-50c3-4bce-b762-36d075f0ce75 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:49.920096 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c670a2-50c3-4bce-b762-36d075f0ce75 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:49.924633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab085a-a40a-4856-84d4-f12df481afb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:49.925214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab085a-a40a-4856-84d4-f12df481afb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:49.925490 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab085a-a40a-4856-84d4-f12df481afb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:49.925755 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab085a-a40a-4856-84d4-f12df481afb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:49.926424 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab085a-a40a-4856-84d4-f12df481afb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:37:49.926732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab085a-a40a-4856-84d4-f12df481afb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:49.928306 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab085a-a40a-4856-84d4-f12df481afb5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:49.940562 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1332/2672] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:37:49.944610 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0104ca4-c59d-4d79-a900-9aa92baff3f1 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:49.954438 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0104ca4-c59d-4d79-a900-9aa92baff3f1 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:49.959039 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2af1fe-3114-4673-ba26-78eef26998e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:49.959616 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2af1fe-3114-4673-ba26-78eef26998e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:49.959899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2af1fe-3114-4673-ba26-78eef26998e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:49.960166 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2af1fe-3114-4673-ba26-78eef26998e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:49.960791 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2af1fe-3114-4673-ba26-78eef26998e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:49.962716 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2af1fe-3114-4673-ba26-78eef26998e4 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:49.988560 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1341/2673] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:37:49 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:37:49.992599 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab085a-a40a-4856-84d4-f12df481afb5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:50.003298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab085a-a40a-4856-84d4-f12df481afb5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:50.058064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32632394-27de-4595-9073-f787eb66b3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:50.058686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32632394-27de-4595-9073-f787eb66b3d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:50.058968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32632394-27de-4595-9073-f787eb66b3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:50.059237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32632394-27de-4595-9073-f787eb66b3d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:50.059844 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32632394-27de-4595-9073-f787eb66b3d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:50.061740 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32632394-27de-4595-9073-f787eb66b3d3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:50.083227 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1333/2674] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:37:49 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:37:50.087333 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2af1fe-3114-4673-ba26-78eef26998e4 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:50.097370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2af1fe-3114-4673-ba26-78eef26998e4 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:50.102106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc72810-89ba-4a65-af44-5d16c121b2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:50.102718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc72810-89ba-4a65-af44-5d16c121b2ac admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:50.102992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc72810-89ba-4a65-af44-5d16c121b2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:50.103257 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc72810-89ba-4a65-af44-5d16c121b2ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:50.103878 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc72810-89ba-4a65-af44-5d16c121b2ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:50.105435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc72810-89ba-4a65-af44-5d16c121b2ac admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:50.115080 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1342/2675] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:37:50.119600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32632394-27de-4595-9073-f787eb66b3d3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:50.129824 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32632394-27de-4595-9073-f787eb66b3d3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:50.135095 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7d04a-b69b-4617-b10a-3b8bdb982a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:50.135680 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7d04a-b69b-4617-b10a-3b8bdb982a89 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:50.135960 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7d04a-b69b-4617-b10a-3b8bdb982a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:50.136229 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7d04a-b69b-4617-b10a-3b8bdb982a89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:50.136871 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7d04a-b69b-4617-b10a-3b8bdb982a89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:50.138485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7d04a-b69b-4617-b10a-3b8bdb982a89 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:50.148890 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1334/2676] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:37:50.153078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc72810-89ba-4a65-af44-5d16c121b2ac admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:50.162959 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc72810-89ba-4a65-af44-5d16c121b2ac admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:50.167557 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd2e10-8b23-4f68-931f-2f3bf3ce416a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:50.168181 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd2e10-8b23-4f68-931f-2f3bf3ce416a admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:50.168452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd2e10-8b23-4f68-931f-2f3bf3ce416a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:50.168719 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd2e10-8b23-4f68-931f-2f3bf3ce416a admin admin] PATH_INFO: `/v3/projects/777198b8b0dc45a4b90739a5bddcef23/users/e9ddc2b9b8a242afacd6fb5325e9de15/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:50.183238 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd2e10-8b23-4f68-931f-2f3bf3ce416a admin admin] RBAC: Authorizing `identity:create_grant(project_id=777198b8b0dc45a4b90739a5bddcef23, user_id=e9ddc2b9b8a242afacd6fb5325e9de15, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:50.185846 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd2e10-8b23-4f68-931f-2f3bf3ce416a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:50.199355 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1343/2677] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:37:50 2022] PUT /identity/v3/projects/777198b8b0dc45a4b90739a5bddcef23/users/e9ddc2b9b8a242afacd6fb5325e9de15/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:37:50.209629 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cc718-2832-47d7-811f-dd67425d3d75 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:50.209918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cc718-2832-47d7-811f-dd67425d3d75 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:50.210191 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cc718-2832-47d7-811f-dd67425d3d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:50.237136 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b22cc718-2832-47d7-811f-dd67425d3d75 None None] MFA Rules not processed for user `e9ddc2b9b8a242afacd6fb5325e9de15`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:50.238420 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b22cc718-2832-47d7-811f-dd67425d3d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:50.301199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1335/2678] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:37:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:50.312809 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd2e10-8b23-4f68-931f-2f3bf3ce416a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:50.314568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-93dd2e10-8b23-4f68-931f-2f3bf3ce416a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:50.370431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd2e10-8b23-4f68-931f-2f3bf3ce416a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:50.374031 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efb537-4499-4018-a320-e73712cfabe4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7776a6a1faf34b369e399d83bb3e18c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:50.374652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efb537-4499-4018-a320-e73712cfabe4 service glance] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:50.374926 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efb537-4499-4018-a320-e73712cfabe4 service glance] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:50.375194 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efb537-4499-4018-a320-e73712cfabe4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:50.384078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efb537-4499-4018-a320-e73712cfabe4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:50.385529 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efb537-4499-4018-a320-e73712cfabe4 service glance] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:50.395384 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1344/2679] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:38:00.298104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:38:00.391366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:38:03.093593 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda5077-fc87-4f53-95a2-35a47a522c9f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:03.094071 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda5077-fc87-4f53-95a2-35a47a522c9f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:03.094179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda5077-fc87-4f53-95a2-35a47a522c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:03.112550 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-acda5077-fc87-4f53-95a2-35a47a522c9f None None] MFA Rules not processed for user `72485bc9ed6f4684b77b617597f99e29`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:38:03.113720 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-acda5077-fc87-4f53-95a2-35a47a522c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:03.165663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1336/2680] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:38:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:38:03.475693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efb537-4499-4018-a320-e73712cfabe4 service glance] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:03.477294 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-40efb537-4499-4018-a320-e73712cfabe4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:03.533397 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efb537-4499-4018-a320-e73712cfabe4 service glance] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:03.537017 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea622b-1fd6-4fe7-be30-073fb77247b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:03.537609 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea622b-1fd6-4fe7-be30-073fb77247b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:03.537885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea622b-1fd6-4fe7-be30-073fb77247b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:03.538156 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea622b-1fd6-4fe7-be30-073fb77247b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:03.539941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-32ea622b-1fd6-4fe7-be30-073fb77247b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:03.549406 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea622b-1fd6-4fe7-be30-073fb77247b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:03.550943 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea622b-1fd6-4fe7-be30-073fb77247b6 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:03.559417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1345/2681] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:38:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:38:13.159876 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=25 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:38:13.555291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=26 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:38:19.549870 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda5077-fc87-4f53-95a2-35a47a522c9f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:19.611243 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda5077-fc87-4f53-95a2-35a47a522c9f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:19.615696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de7e76-1a1f-46ae-8a22-624a509b5b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:19.616309 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de7e76-1a1f-46ae-8a22-624a509b5b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:19.616589 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de7e76-1a1f-46ae-8a22-624a509b5b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:19.616862 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de7e76-1a1f-46ae-8a22-624a509b5b50 admin admin] PATH_INFO: `/v3/users/e9ddc2b9b8a242afacd6fb5325e9de15` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:19.617969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de7e76-1a1f-46ae-8a22-624a509b5b50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9ddc2b9b8a242afacd6fb5325e9de15)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:19.619966 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de7e76-1a1f-46ae-8a22-624a509b5b50 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:19.666273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f2de7e76-1a1f-46ae-8a22-624a509b5b50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9ddc2b9b8a242afacd6fb5325e9de15'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:19.673168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f2de7e76-1a1f-46ae-8a22-624a509b5b50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9ddc2b9b8a242afacd6fb5325e9de15'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:19.680157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f2de7e76-1a1f-46ae-8a22-624a509b5b50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9ddc2b9b8a242afacd6fb5325e9de15'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:19.680787 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f2de7e76-1a1f-46ae-8a22-624a509b5b50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9ddc2b9b8a242afacd6fb5325e9de15'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:19.681412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f2de7e76-1a1f-46ae-8a22-624a509b5b50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9ddc2b9b8a242afacd6fb5325e9de15'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:19.698959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1337/2682] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:38:19 2022] DELETE /identity/v3/users/e9ddc2b9b8a242afacd6fb5325e9de15 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:38:19.703308 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f9e02-7d02-4354-803b-7fd8d236f31b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:19.703792 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f9e02-7d02-4354-803b-7fd8d236f31b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:19.703904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f9e02-7d02-4354-803b-7fd8d236f31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:19.722752 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-948f9e02-7d02-4354-803b-7fd8d236f31b None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:38:19.723952 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-948f9e02-7d02-4354-803b-7fd8d236f31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:19.785081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1346/2683] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:38:19.794771 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de7e76-1a1f-46ae-8a22-624a509b5b50 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:19.796232 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f2de7e76-1a1f-46ae-8a22-624a509b5b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:19.852234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de7e76-1a1f-46ae-8a22-624a509b5b50 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:19.855890 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ee2d9-e7dd-4583-abac-5a2ad09882aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:19.856479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ee2d9-e7dd-4583-abac-5a2ad09882aa service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:19.856754 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ee2d9-e7dd-4583-abac-5a2ad09882aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:19.857021 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ee2d9-e7dd-4583-abac-5a2ad09882aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:19.865809 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ee2d9-e7dd-4583-abac-5a2ad09882aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:19.867255 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ee2d9-e7dd-4583-abac-5a2ad09882aa service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:19.876997 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1338/2684] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:38:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:38:20.144095 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f9e02-7d02-4354-803b-7fd8d236f31b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:20.145598 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-948f9e02-7d02-4354-803b-7fd8d236f31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:20.158779 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f9e02-7d02-4354-803b-7fd8d236f31b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:20.164121 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b26ca-2841-4c8e-b3b8-5b3cc315a7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:20.164745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b26ca-2841-4c8e-b3b8-5b3cc315a7e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:20.165019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b26ca-2841-4c8e-b3b8-5b3cc315a7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:20.165286 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b26ca-2841-4c8e-b3b8-5b3cc315a7e5 admin admin] PATH_INFO: `/v3/projects/777198b8b0dc45a4b90739a5bddcef23` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:20.166476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b26ca-2841-4c8e-b3b8-5b3cc315a7e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=777198b8b0dc45a4b90739a5bddcef23)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:20.168367 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b26ca-2841-4c8e-b3b8-5b3cc315a7e5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:20.222732 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1f5b26ca-2841-4c8e-b3b8-5b3cc315a7e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '777198b8b0dc45a4b90739a5bddcef23', 'request_id': 'req-1f5b26ca-2841-4c8e-b3b8-5b3cc315a7e5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:20.228121 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1f5b26ca-2841-4c8e-b3b8-5b3cc315a7e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '777198b8b0dc45a4b90739a5bddcef23', 'request_id': 'req-1f5b26ca-2841-4c8e-b3b8-5b3cc315a7e5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:20.235072 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1f5b26ca-2841-4c8e-b3b8-5b3cc315a7e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '777198b8b0dc45a4b90739a5bddcef23', 'request_id': 'req-1f5b26ca-2841-4c8e-b3b8-5b3cc315a7e5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:20.246399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1f5b26ca-2841-4c8e-b3b8-5b3cc315a7e5 admin admin] The token cache is being invalidate because project 777198b8b0dc45a4b90739a5bddcef23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:38:20.246685 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1f5b26ca-2841-4c8e-b3b8-5b3cc315a7e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:20.247311 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1f5b26ca-2841-4c8e-b3b8-5b3cc315a7e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:20.248775 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1347/2685] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:38:20 2022] DELETE /identity/v3/projects/777198b8b0dc45a4b90739a5bddcef23 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:38:20.253525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1f6df-8b78-492e-846d-0b77f57036fa None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:20.253808 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1f6df-8b78-492e-846d-0b77f57036fa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:20.254110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1f6df-8b78-492e-846d-0b77f57036fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:20.273559 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-07d1f6df-8b78-492e-846d-0b77f57036fa None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:38:20.274790 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-07d1f6df-8b78-492e-846d-0b77f57036fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:20.291965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:38:20.337060 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1339/2686] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:38:20.345098 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e106d-2639-4373-806a-22923a0e298c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:20.345385 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e106d-2639-4373-806a-22923a0e298c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:20.345667 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e106d-2639-4373-806a-22923a0e298c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:20.366821 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-9c0e106d-2639-4373-806a-22923a0e298c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:38:20.368112 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9c0e106d-2639-4373-806a-22923a0e298c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:20.377140 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1348/2687] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:38:20.381566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1f6df-8b78-492e-846d-0b77f57036fa None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:20.391240 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1f6df-8b78-492e-846d-0b77f57036fa None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:20.395848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fed06f-b41f-4d64-98c3-0be0b5170ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:20.396440 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fed06f-b41f-4d64-98c3-0be0b5170ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:20.396719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fed06f-b41f-4d64-98c3-0be0b5170ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:20.396984 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fed06f-b41f-4d64-98c3-0be0b5170ac3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:20.397621 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fed06f-b41f-4d64-98c3-0be0b5170ac3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:38:20.397923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fed06f-b41f-4d64-98c3-0be0b5170ac3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:20.399806 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fed06f-b41f-4d64-98c3-0be0b5170ac3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:20.411909 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1340/2688] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:38:20.417190 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e106d-2639-4373-806a-22923a0e298c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:20.427064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e106d-2639-4373-806a-22923a0e298c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:20.432405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ec742-ccfb-4fa9-bf52-7b9f2e03a2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:20.433004 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ec742-ccfb-4fa9-bf52-7b9f2e03a2b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:20.433282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ec742-ccfb-4fa9-bf52-7b9f2e03a2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:20.433553 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ec742-ccfb-4fa9-bf52-7b9f2e03a2b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:20.434181 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ec742-ccfb-4fa9-bf52-7b9f2e03a2b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:20.436222 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ec742-ccfb-4fa9-bf52-7b9f2e03a2b2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:20.464914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1349/2689] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:38:20 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:38:20.468829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fed06f-b41f-4d64-98c3-0be0b5170ac3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:20.479691 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fed06f-b41f-4d64-98c3-0be0b5170ac3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:20.533791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550740b-2eb2-4996-ba12-b728f94a6aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:20.534442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550740b-2eb2-4996-ba12-b728f94a6aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:20.534723 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550740b-2eb2-4996-ba12-b728f94a6aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:20.534993 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550740b-2eb2-4996-ba12-b728f94a6aa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:20.535596 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550740b-2eb2-4996-ba12-b728f94a6aa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:20.537510 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550740b-2eb2-4996-ba12-b728f94a6aa9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:20.559168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1341/2690] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:38:20 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:38:20.563187 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ec742-ccfb-4fa9-bf52-7b9f2e03a2b2 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:20.573558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ec742-ccfb-4fa9-bf52-7b9f2e03a2b2 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:20.578157 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36af14-981c-4a07-8b88-f5aba2f03915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:20.578793 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36af14-981c-4a07-8b88-f5aba2f03915 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:20.579073 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36af14-981c-4a07-8b88-f5aba2f03915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:20.579343 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36af14-981c-4a07-8b88-f5aba2f03915 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:20.579988 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36af14-981c-4a07-8b88-f5aba2f03915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:20.581563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36af14-981c-4a07-8b88-f5aba2f03915 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:20.592415 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1350/2691] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:38:20.596751 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550740b-2eb2-4996-ba12-b728f94a6aa9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:20.608078 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550740b-2eb2-4996-ba12-b728f94a6aa9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:20.612913 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fe2de-4463-48be-9498-0225a64fdd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:20.613513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fe2de-4463-48be-9498-0225a64fdd63 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:20.613785 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fe2de-4463-48be-9498-0225a64fdd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:20.614053 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fe2de-4463-48be-9498-0225a64fdd63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:20.614745 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fe2de-4463-48be-9498-0225a64fdd63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:20.616329 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fe2de-4463-48be-9498-0225a64fdd63 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:20.630896 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1342/2692] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:38:20.635475 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36af14-981c-4a07-8b88-f5aba2f03915 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:20.645614 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36af14-981c-4a07-8b88-f5aba2f03915 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:20.650797 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505d1be-bd29-469e-b932-41526e9980f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:20.651505 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505d1be-bd29-469e-b932-41526e9980f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:20.651789 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505d1be-bd29-469e-b932-41526e9980f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:20.652064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505d1be-bd29-469e-b932-41526e9980f5 admin admin] PATH_INFO: `/v3/projects/87a920807a5c494ca0eab180037b1f80/users/7f5ce64eb603454ebc3e5e5b312ae09e/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:20.667232 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505d1be-bd29-469e-b932-41526e9980f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87a920807a5c494ca0eab180037b1f80, user_id=7f5ce64eb603454ebc3e5e5b312ae09e, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:20.669859 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505d1be-bd29-469e-b932-41526e9980f5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:20.682536 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1351/2693] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:38:20 2022] PUT /identity/v3/projects/87a920807a5c494ca0eab180037b1f80/users/7f5ce64eb603454ebc3e5e5b312ae09e/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:38:20.694163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6daf7-dbfd-496e-b4c2-cf1777b56a97 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:20.694701 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6daf7-dbfd-496e-b4c2-cf1777b56a97 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:20.694969 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6daf7-dbfd-496e-b4c2-cf1777b56a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:20.724458 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9db6daf7-dbfd-496e-b4c2-cf1777b56a97 None None] MFA Rules not processed for user `7f5ce64eb603454ebc3e5e5b312ae09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:38:20.725747 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9db6daf7-dbfd-496e-b4c2-cf1777b56a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:20.794619 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1343/2694] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:38:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:38:20.806597 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505d1be-bd29-469e-b932-41526e9980f5 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:20.808407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8505d1be-bd29-469e-b932-41526e9980f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:20.830890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:38:20.870805 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505d1be-bd29-469e-b932-41526e9980f5 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:20.874551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b638ba6-834d-461f-9cc9-67b4d4c372ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:20.875143 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b638ba6-834d-461f-9cc9-67b4d4c372ac service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:20.875418 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b638ba6-834d-461f-9cc9-67b4d4c372ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:20.875686 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b638ba6-834d-461f-9cc9-67b4d4c372ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:20.883945 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b638ba6-834d-461f-9cc9-67b4d4c372ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:20.885392 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b638ba6-834d-461f-9cc9-67b4d4c372ac service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:20.895935 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1352/2695] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:38:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:38:22.988182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6daf7-dbfd-496e-b4c2-cf1777b56a97 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:22.989601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9db6daf7-dbfd-496e-b4c2-cf1777b56a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:23.009567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:38:23.046896 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6daf7-dbfd-496e-b4c2-cf1777b56a97 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:23.050422 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a952e85-7119-45a1-9ec5-8c6ffa1199ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:23.051013 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a952e85-7119-45a1-9ec5-8c6ffa1199ab service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:23.051285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a952e85-7119-45a1-9ec5-8c6ffa1199ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:23.051546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a952e85-7119-45a1-9ec5-8c6ffa1199ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:23.053234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7a952e85-7119-45a1-9ec5-8c6ffa1199ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:23.113456 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a952e85-7119-45a1-9ec5-8c6ffa1199ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:23.114919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a952e85-7119-45a1-9ec5-8c6ffa1199ab service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:23.124278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1344/2696] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:38:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:38:30.290155 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b638ba6-834d-461f-9cc9-67b4d4c372ac service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:30.350167 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b638ba6-834d-461f-9cc9-67b4d4c372ac service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:30.355155 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c6282-140b-411d-bdcf-266a92204d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:30.355755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c6282-140b-411d-bdcf-266a92204d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:30.356029 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c6282-140b-411d-bdcf-266a92204d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:30.356296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c6282-140b-411d-bdcf-266a92204d16 admin admin] PATH_INFO: `/v3/users/7f5ce64eb603454ebc3e5e5b312ae09e` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:30.357387 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c6282-140b-411d-bdcf-266a92204d16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f5ce64eb603454ebc3e5e5b312ae09e)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:30.359307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c6282-140b-411d-bdcf-266a92204d16 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:30.404642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-dd2c6282-140b-411d-bdcf-266a92204d16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f5ce64eb603454ebc3e5e5b312ae09e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:30.411697 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-dd2c6282-140b-411d-bdcf-266a92204d16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f5ce64eb603454ebc3e5e5b312ae09e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:30.421091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-dd2c6282-140b-411d-bdcf-266a92204d16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f5ce64eb603454ebc3e5e5b312ae09e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:30.421773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-dd2c6282-140b-411d-bdcf-266a92204d16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f5ce64eb603454ebc3e5e5b312ae09e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:30.422438 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-dd2c6282-140b-411d-bdcf-266a92204d16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f5ce64eb603454ebc3e5e5b312ae09e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:30.440435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1353/2697] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:38:30 2022] DELETE /identity/v3/users/7f5ce64eb603454ebc3e5e5b312ae09e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:38:30.445107 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17738c-38d2-4693-8cfa-9b249eb7a8d7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:30.445591 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17738c-38d2-4693-8cfa-9b249eb7a8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:30.445591 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17738c-38d2-4693-8cfa-9b249eb7a8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:30.464685 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-de17738c-38d2-4693-8cfa-9b249eb7a8d7 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:38:30.465885 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-de17738c-38d2-4693-8cfa-9b249eb7a8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:30.523379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1345/2698] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:38:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:38:30.533446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c6282-140b-411d-bdcf-266a92204d16 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:30.534865 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dd2c6282-140b-411d-bdcf-266a92204d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:30.580547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:38:30.594439 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c6282-140b-411d-bdcf-266a92204d16 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:30.598151 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe24ef-a4e8-4f6b-927f-7f56779d4d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:30.598783 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe24ef-a4e8-4f6b-927f-7f56779d4d75 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:30.599064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe24ef-a4e8-4f6b-927f-7f56779d4d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:30.599335 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe24ef-a4e8-4f6b-927f-7f56779d4d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:30.608734 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe24ef-a4e8-4f6b-927f-7f56779d4d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:30.610157 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe24ef-a4e8-4f6b-927f-7f56779d4d75 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:30.620049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1354/2699] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:38:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:38:30.903453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17738c-38d2-4693-8cfa-9b249eb7a8d7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:30.904929 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-de17738c-38d2-4693-8cfa-9b249eb7a8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:30.919414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17738c-38d2-4693-8cfa-9b249eb7a8d7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:30.924041 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327dc53-85a5-4eff-8566-fbd2372b4634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:30.924665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327dc53-85a5-4eff-8566-fbd2372b4634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:30.924942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327dc53-85a5-4eff-8566-fbd2372b4634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:30.925212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327dc53-85a5-4eff-8566-fbd2372b4634 admin admin] PATH_INFO: `/v3/projects/87a920807a5c494ca0eab180037b1f80` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:30.926347 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327dc53-85a5-4eff-8566-fbd2372b4634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87a920807a5c494ca0eab180037b1f80)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:30.928254 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327dc53-85a5-4eff-8566-fbd2372b4634 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:30.974416 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c327dc53-85a5-4eff-8566-fbd2372b4634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87a920807a5c494ca0eab180037b1f80', 'request_id': 'req-c327dc53-85a5-4eff-8566-fbd2372b4634'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:30.979797 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c327dc53-85a5-4eff-8566-fbd2372b4634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87a920807a5c494ca0eab180037b1f80', 'request_id': 'req-c327dc53-85a5-4eff-8566-fbd2372b4634'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:30.983592 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:38:30.987500 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c327dc53-85a5-4eff-8566-fbd2372b4634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87a920807a5c494ca0eab180037b1f80', 'request_id': 'req-c327dc53-85a5-4eff-8566-fbd2372b4634'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:31.000828 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c327dc53-85a5-4eff-8566-fbd2372b4634 admin admin] The token cache is being invalidate because project 87a920807a5c494ca0eab180037b1f80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:38:31.001172 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c327dc53-85a5-4eff-8566-fbd2372b4634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:31.001852 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c327dc53-85a5-4eff-8566-fbd2372b4634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:31.003386 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1346/2700] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:38:30 2022] DELETE /identity/v3/projects/87a920807a5c494ca0eab180037b1f80 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:38:31.008544 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06c1be-8a12-4c7a-9a94-42f2784150a1 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:31.008831 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06c1be-8a12-4c7a-9a94-42f2784150a1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:31.009104 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06c1be-8a12-4c7a-9a94-42f2784150a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:31.029553 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ce06c1be-8a12-4c7a-9a94-42f2784150a1 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:38:31.030730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ce06c1be-8a12-4c7a-9a94-42f2784150a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:31.089207 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1355/2701] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:38:31.096938 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e174e44-5e13-40fe-8a6e-2191d282ddab None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:31.097251 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e174e44-5e13-40fe-8a6e-2191d282ddab None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:31.097527 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e174e44-5e13-40fe-8a6e-2191d282ddab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:31.116732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2e174e44-5e13-40fe-8a6e-2191d282ddab None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:38:31.117936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2e174e44-5e13-40fe-8a6e-2191d282ddab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:31.126637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1347/2702] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:38:31.130830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06c1be-8a12-4c7a-9a94-42f2784150a1 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:31.141238 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06c1be-8a12-4c7a-9a94-42f2784150a1 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:31.145805 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66940fab-312c-4ba3-9d61-b6ac888c523b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:31.146427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66940fab-312c-4ba3-9d61-b6ac888c523b admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:31.146706 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66940fab-312c-4ba3-9d61-b6ac888c523b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:31.146978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66940fab-312c-4ba3-9d61-b6ac888c523b admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:31.147642 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66940fab-312c-4ba3-9d61-b6ac888c523b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:38:31.147954 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66940fab-312c-4ba3-9d61-b6ac888c523b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:31.149630 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66940fab-312c-4ba3-9d61-b6ac888c523b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:31.160905 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1356/2703] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:38:31.165005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e174e44-5e13-40fe-8a6e-2191d282ddab None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:31.174163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e174e44-5e13-40fe-8a6e-2191d282ddab None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:31.179341 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4ad72-e09d-4d35-8dfa-a0793fc63556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:31.179921 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4ad72-e09d-4d35-8dfa-a0793fc63556 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:31.180200 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4ad72-e09d-4d35-8dfa-a0793fc63556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:31.180469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4ad72-e09d-4d35-8dfa-a0793fc63556 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:31.181591 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c4ad72-e09d-4d35-8dfa-a0793fc63556 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:31.183505 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c4ad72-e09d-4d35-8dfa-a0793fc63556 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:31.211113 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1348/2704] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:38:31 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:38:31.215290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66940fab-312c-4ba3-9d61-b6ac888c523b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:31.225529 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66940fab-312c-4ba3-9d61-b6ac888c523b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:31.280512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a76cf-313b-4f8c-89e3-1e8f19718b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:31.281034 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a76cf-313b-4f8c-89e3-1e8f19718b2b admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:31.281310 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a76cf-313b-4f8c-89e3-1e8f19718b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:31.281581 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a76cf-313b-4f8c-89e3-1e8f19718b2b admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:31.282214 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a76cf-313b-4f8c-89e3-1e8f19718b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:31.284129 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a76cf-313b-4f8c-89e3-1e8f19718b2b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:31.304149 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1357/2705] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:38:31 2022] POST /identity/v3/users => generated 286 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:38:31.308889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4ad72-e09d-4d35-8dfa-a0793fc63556 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:31.319298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4ad72-e09d-4d35-8dfa-a0793fc63556 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:31.324473 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95770a96-208e-4edf-8b54-30d62a22bef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:31.325050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95770a96-208e-4edf-8b54-30d62a22bef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:31.325323 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95770a96-208e-4edf-8b54-30d62a22bef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:31.325588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95770a96-208e-4edf-8b54-30d62a22bef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:31.326224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95770a96-208e-4edf-8b54-30d62a22bef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:31.327828 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95770a96-208e-4edf-8b54-30d62a22bef3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:31.337403 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1349/2706] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:38:31.341206 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a76cf-313b-4f8c-89e3-1e8f19718b2b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:31.351318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a76cf-313b-4f8c-89e3-1e8f19718b2b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:31.356002 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2eec97-1b83-4ce9-81ad-684eb2626e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:31.356593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2eec97-1b83-4ce9-81ad-684eb2626e81 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:31.356873 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2eec97-1b83-4ce9-81ad-684eb2626e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:31.357146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2eec97-1b83-4ce9-81ad-684eb2626e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:31.357817 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2eec97-1b83-4ce9-81ad-684eb2626e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:31.359430 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2eec97-1b83-4ce9-81ad-684eb2626e81 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:31.370394 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1358/2707] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:38:31.374575 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95770a96-208e-4edf-8b54-30d62a22bef3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:31.384925 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95770a96-208e-4edf-8b54-30d62a22bef3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:31.389484 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c44bfb-885c-48e7-863a-a89e8b25991d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:31.390109 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c44bfb-885c-48e7-863a-a89e8b25991d admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:31.390420 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c44bfb-885c-48e7-863a-a89e8b25991d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:31.390688 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c44bfb-885c-48e7-863a-a89e8b25991d admin admin] PATH_INFO: `/v3/projects/00a10f60283c4e379ac0c72b6b6aa462/users/946e032317384ffc8ce0778cb9b1e39f/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:31.405459 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c44bfb-885c-48e7-863a-a89e8b25991d admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a10f60283c4e379ac0c72b6b6aa462, user_id=946e032317384ffc8ce0778cb9b1e39f, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:31.408144 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c44bfb-885c-48e7-863a-a89e8b25991d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:31.421567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1350/2708] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:38:31 2022] PUT /identity/v3/projects/00a10f60283c4e379ac0c72b6b6aa462/users/946e032317384ffc8ce0778cb9b1e39f/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:38:31.487285 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe97512-8e23-4c25-bd05-23708ae779fc None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:31.487574 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe97512-8e23-4c25-bd05-23708ae779fc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:31.487850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe97512-8e23-4c25-bd05-23708ae779fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:31.515865 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ffe97512-8e23-4c25-bd05-23708ae779fc None None] MFA Rules not processed for user `946e032317384ffc8ce0778cb9b1e39f`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:38:31.517075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ffe97512-8e23-4c25-bd05-23708ae779fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:31.559806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:38:31.585292 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1359/2709] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:38:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:38:31.596611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c44bfb-885c-48e7-863a-a89e8b25991d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:31.598310 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-76c44bfb-885c-48e7-863a-a89e8b25991d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:31.674255 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c44bfb-885c-48e7-863a-a89e8b25991d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:31.677849 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d7301-49af-4296-8694-6f5fae969912 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7776a6a1faf34b369e399d83bb3e18c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:31.678467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d7301-49af-4296-8694-6f5fae969912 service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:31.678741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d7301-49af-4296-8694-6f5fae969912 service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:31.679008 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d7301-49af-4296-8694-6f5fae969912 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:31.687625 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d7301-49af-4296-8694-6f5fae969912 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:31.689024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d7301-49af-4296-8694-6f5fae969912 service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:31.697998 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1351/2710] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:38:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:38:41.582435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=9 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:38:41.694163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=66,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:39:02.073222 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe97512-8e23-4c25-bd05-23708ae779fc None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:39:02.076409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ffe97512-8e23-4c25-bd05-23708ae779fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:39:02.153027 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe97512-8e23-4c25-bd05-23708ae779fc None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:39:02.158419 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f343ec8-656d-4c44-83f1-282114399c4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:39:02.159792 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f343ec8-656d-4c44-83f1-282114399c4c service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:39:02.160019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f343ec8-656d-4c44-83f1-282114399c4c service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:39:02.160353 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f343ec8-656d-4c44-83f1-282114399c4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:39:02.162140 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8f343ec8-656d-4c44-83f1-282114399c4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:39:02.218788 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f343ec8-656d-4c44-83f1-282114399c4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:39:02.220347 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f343ec8-656d-4c44-83f1-282114399c4c service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:39:02.229995 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1360/2711] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:39:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:39:05.533005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f523d-c196-4996-9630-f4434af18429 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:39:05.533535 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f523d-c196-4996-9630-f4434af18429 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:39:05.533637 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f523d-c196-4996-9630-f4434af18429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:39:05.552237 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d14f523d-c196-4996-9630-f4434af18429 None None] MFA Rules not processed for user `5de4e28bfee14bae93c7f7a8f3f098fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:39:05.553436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d14f523d-c196-4996-9630-f4434af18429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:39:05.560473 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1352/2712] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:39:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:39:07.836189 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f343ec8-656d-4c44-83f1-282114399c4c service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:39:07.897552 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f343ec8-656d-4c44-83f1-282114399c4c service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:39:07.901985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea0b35-d14f-4b18-a2b1-8f36960cc3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:39:07.902624 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea0b35-d14f-4b18-a2b1-8f36960cc3d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:39:07.902899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea0b35-d14f-4b18-a2b1-8f36960cc3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:39:07.903167 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea0b35-d14f-4b18-a2b1-8f36960cc3d8 admin admin] PATH_INFO: `/v3/users/946e032317384ffc8ce0778cb9b1e39f` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:39:07.904291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea0b35-d14f-4b18-a2b1-8f36960cc3d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=946e032317384ffc8ce0778cb9b1e39f)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:39:07.906167 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea0b35-d14f-4b18-a2b1-8f36960cc3d8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:39:07.946399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:39:07.952920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f3ea0b35-d14f-4b18-a2b1-8f36960cc3d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '946e032317384ffc8ce0778cb9b1e39f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:39:07.959588 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f3ea0b35-d14f-4b18-a2b1-8f36960cc3d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '946e032317384ffc8ce0778cb9b1e39f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:39:07.966576 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f3ea0b35-d14f-4b18-a2b1-8f36960cc3d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '946e032317384ffc8ce0778cb9b1e39f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:39:07.967311 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f3ea0b35-d14f-4b18-a2b1-8f36960cc3d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '946e032317384ffc8ce0778cb9b1e39f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:39:07.967971 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f3ea0b35-d14f-4b18-a2b1-8f36960cc3d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '946e032317384ffc8ce0778cb9b1e39f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:39:07.985396 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1361/2713] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:39:07 2022] DELETE /identity/v3/users/946e032317384ffc8ce0778cb9b1e39f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:39:07.990460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea1671-5289-42c9-a45c-a057499c56ad None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:39:07.990932 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea1671-5289-42c9-a45c-a057499c56ad None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:39:07.991050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea1671-5289-42c9-a45c-a057499c56ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:39:08.009272 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-12ea1671-5289-42c9-a45c-a057499c56ad None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:39:08.010494 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-12ea1671-5289-42c9-a45c-a057499c56ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:39:08.067639 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1353/2714] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:39:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:39:08.077179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea0b35-d14f-4b18-a2b1-8f36960cc3d8 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:39:08.078620 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f3ea0b35-d14f-4b18-a2b1-8f36960cc3d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:39:08.135910 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea0b35-d14f-4b18-a2b1-8f36960cc3d8 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:39:08.139569 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b867441-9c4d-4a66-8723-5391c2b3d02f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:39:08.140161 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b867441-9c4d-4a66-8723-5391c2b3d02f service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:39:08.140438 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b867441-9c4d-4a66-8723-5391c2b3d02f service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:39:08.140709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b867441-9c4d-4a66-8723-5391c2b3d02f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:39:08.150239 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b867441-9c4d-4a66-8723-5391c2b3d02f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:39:08.151682 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b867441-9c4d-4a66-8723-5391c2b3d02f service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:39:08.161617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1362/2715] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:39:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:39:08.459259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea1671-5289-42c9-a45c-a057499c56ad None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:39:08.460835 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-12ea1671-5289-42c9-a45c-a057499c56ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:39:08.475862 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea1671-5289-42c9-a45c-a057499c56ad None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:39:08.482174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92cb9f-feba-4df8-983b-a76a3f36f10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:39:08.483210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92cb9f-feba-4df8-983b-a76a3f36f10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:39:08.483460 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92cb9f-feba-4df8-983b-a76a3f36f10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:39:08.483731 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92cb9f-feba-4df8-983b-a76a3f36f10c admin admin] PATH_INFO: `/v3/projects/00a10f60283c4e379ac0c72b6b6aa462` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:39:08.485070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92cb9f-feba-4df8-983b-a76a3f36f10c admin admin] RBAC: Authorizing `identity:delete_project(project_id=00a10f60283c4e379ac0c72b6b6aa462)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:39:08.486992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92cb9f-feba-4df8-983b-a76a3f36f10c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:39:08.533287 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4f92cb9f-feba-4df8-983b-a76a3f36f10c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00a10f60283c4e379ac0c72b6b6aa462', 'request_id': 'req-4f92cb9f-feba-4df8-983b-a76a3f36f10c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:39:08.538999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4f92cb9f-feba-4df8-983b-a76a3f36f10c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00a10f60283c4e379ac0c72b6b6aa462', 'request_id': 'req-4f92cb9f-feba-4df8-983b-a76a3f36f10c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:39:08.548926 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4f92cb9f-feba-4df8-983b-a76a3f36f10c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00a10f60283c4e379ac0c72b6b6aa462', 'request_id': 'req-4f92cb9f-feba-4df8-983b-a76a3f36f10c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:39:08.560853 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4f92cb9f-feba-4df8-983b-a76a3f36f10c admin admin] The token cache is being invalidate because project 00a10f60283c4e379ac0c72b6b6aa462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:39:08.561139 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4f92cb9f-feba-4df8-983b-a76a3f36f10c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:39:08.561773 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4f92cb9f-feba-4df8-983b-a76a3f36f10c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:39:08.563311 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1354/2716] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:39:08 2022] DELETE /identity/v3/projects/00a10f60283c4e379ac0c72b6b6aa462 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:39:18.156874 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=5,keystone:SELECT=40,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:39:18.557829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:39:40.698806 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b867441-9c4d-4a66-8723-5391c2b3d02f service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:39:40.700158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2b867441-9c4d-4a66-8723-5391c2b3d02f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:39:40.766794 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b867441-9c4d-4a66-8723-5391c2b3d02f service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:39:40.823625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5323cd-fc71-4cfd-8e45-a741f2c43686 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f62745b92a864480a9438de581e1d05e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:39:40.824241 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5323cd-fc71-4cfd-8e45-a741f2c43686 service placement] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:39:40.824518 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5323cd-fc71-4cfd-8e45-a741f2c43686 service placement] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:39:40.824791 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5323cd-fc71-4cfd-8e45-a741f2c43686 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:39:40.826547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5c5323cd-fc71-4cfd-8e45-a741f2c43686 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:39:40.837935 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5323cd-fc71-4cfd-8e45-a741f2c43686 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:39:40.839520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5323cd-fc71-4cfd-8e45-a741f2c43686 service placement] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:39:40.849113 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1363/2717] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:39:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:39:50.844989 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=58 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:40:40.862029 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92cb9f-feba-4df8-983b-a76a3f36f10c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:40:40.864036 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4f92cb9f-feba-4df8-983b-a76a3f36f10c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:40.923157 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92cb9f-feba-4df8-983b-a76a3f36f10c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:40:40.928877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3f9cc-cd1d-4aa9-9f39-b86ca4fd0067 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7776a6a1faf34b369e399d83bb3e18c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:40:40.929476 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3f9cc-cd1d-4aa9-9f39-b86ca4fd0067 service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:40.929757 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3f9cc-cd1d-4aa9-9f39-b86ca4fd0067 service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:40.930024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3f9cc-cd1d-4aa9-9f39-b86ca4fd0067 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:40.931786 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-45a3f9cc-cd1d-4aa9-9f39-b86ca4fd0067 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:41.000914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3f9cc-cd1d-4aa9-9f39-b86ca4fd0067 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:40:41.002268 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3f9cc-cd1d-4aa9-9f39-b86ca4fd0067 service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:40:41.013182 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1355/2718] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:40:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:40:41.173934 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5323cd-fc71-4cfd-8e45-a741f2c43686 service placement] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:40:41.175356 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5c5323cd-fc71-4cfd-8e45-a741f2c43686 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:41.234633 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5323cd-fc71-4cfd-8e45-a741f2c43686 service placement] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:40:41.238174 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be8fc4-b115-4ed3-b172-19d28d192b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:40:41.238797 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be8fc4-b115-4ed3-b172-19d28d192b56 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:41.239075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be8fc4-b115-4ed3-b172-19d28d192b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:41.239341 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be8fc4-b115-4ed3-b172-19d28d192b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:41.241025 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-76be8fc4-b115-4ed3-b172-19d28d192b56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:41.254301 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be8fc4-b115-4ed3-b172-19d28d192b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:40:41.255653 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be8fc4-b115-4ed3-b172-19d28d192b56 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:40:41.266747 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1364/2719] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:40:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:40:51.008442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=54 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:40:51.261714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=26 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:40:58.933676 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56429f-a307-4a28-ab02-5982dc7d8b3d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:58.934226 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56429f-a307-4a28-ab02-5982dc7d8b3d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:58.934226 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56429f-a307-4a28-ab02-5982dc7d8b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:58.953866 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fd56429f-a307-4a28-ab02-5982dc7d8b3d None None] MFA Rules not processed for user `b88ccd3e5c4c48d3a27f4eaa7874e28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:40:58.955055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fd56429f-a307-4a28-ab02-5982dc7d8b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:59.013281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1356/2720] 10.222.0.39 () {58 vars in 1009 bytes} [Mon Aug 8 02:40:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:40:59.059993 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be8fc4-b115-4ed3-b172-19d28d192b56 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:40:59.061420 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-76be8fc4-b115-4ed3-b172-19d28d192b56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:59.127539 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be8fc4-b115-4ed3-b172-19d28d192b56 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:40:59.132583 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473749f-d9b0-44bc-b7f8-99fe9015f5cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac60b4d621234225830c94edfac400e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:40:59.133210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473749f-d9b0-44bc-b7f8-99fe9015f5cf service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:59.133484 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473749f-d9b0-44bc-b7f8-99fe9015f5cf service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:59.133749 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473749f-d9b0-44bc-b7f8-99fe9015f5cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:59.141472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473749f-d9b0-44bc-b7f8-99fe9015f5cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:40:59.143070 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473749f-d9b0-44bc-b7f8-99fe9015f5cf service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:40:59.151488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1365/2721] 10.222.0.39 () {60 vars in 1384 bytes} [Mon Aug 8 02:40:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:41:09.006123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=25 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:09.148396 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=32 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:14.098879 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046ea12-22a6-47a8-8bbe-16e7bcb99063 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:14.099428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046ea12-22a6-47a8-8bbe-16e7bcb99063 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:14.099593 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046ea12-22a6-47a8-8bbe-16e7bcb99063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:14.118984 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d046ea12-22a6-47a8-8bbe-16e7bcb99063 None None] MFA Rules not processed for user `72485bc9ed6f4684b77b617597f99e29`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:14.120172 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d046ea12-22a6-47a8-8bbe-16e7bcb99063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:14.176553 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1357/2722] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:41:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:14.339793 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473749f-d9b0-44bc-b7f8-99fe9015f5cf service swift] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:14.341265 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1473749f-d9b0-44bc-b7f8-99fe9015f5cf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:14.353331 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473749f-d9b0-44bc-b7f8-99fe9015f5cf service swift] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:14.356844 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce570a5-324f-4216-b198-472ca1e1b547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:14.357434 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce570a5-324f-4216-b198-472ca1e1b547 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:14.357714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce570a5-324f-4216-b198-472ca1e1b547 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:14.357992 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce570a5-324f-4216-b198-472ca1e1b547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:14.359724 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4ce570a5-324f-4216-b198-472ca1e1b547 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:14.369342 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce570a5-324f-4216-b198-472ca1e1b547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:14.370862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce570a5-324f-4216-b198-472ca1e1b547 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:14.379827 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1366/2723] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:41:14.509072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b67e9-fb40-4990-a7d4-edc7e5ecdb1c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:14.509676 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b67e9-fb40-4990-a7d4-edc7e5ecdb1c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:14.510050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b67e9-fb40-4990-a7d4-edc7e5ecdb1c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:14.511641 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1358/2724] 10.222.0.39 () {54 vars in 921 bytes} [Mon Aug 8 02:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:41:14.518227 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52c166-6b39-4f2b-8391-f99fa431aff1 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:14.518871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52c166-6b39-4f2b-8391-f99fa431aff1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:14.519243 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52c166-6b39-4f2b-8391-f99fa431aff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:14.540782 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-9c52c166-6b39-4f2b-8391-f99fa431aff1 None None] MFA Rules not processed for user `c4238566a9c248ffbfa314f47f02670c`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:14.542013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9c52c166-6b39-4f2b-8391-f99fa431aff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:14.552727 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1367/2725] 10.222.0.39 () {58 vars in 1018 bytes} [Mon Aug 8 02:41:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:14.566604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b67e9-fb40-4990-a7d4-edc7e5ecdb1c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:14.578720 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b67e9-fb40-4990-a7d4-edc7e5ecdb1c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:14.582531 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a097c1-4cef-4b6a-913b-cb0908a766ad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7776a6a1faf34b369e399d83bb3e18c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:14.583412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a097c1-4cef-4b6a-913b-cb0908a766ad service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:14.583758 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a097c1-4cef-4b6a-913b-cb0908a766ad service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:14.584167 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a097c1-4cef-4b6a-913b-cb0908a766ad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:14.595576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a097c1-4cef-4b6a-913b-cb0908a766ad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:14.598379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a097c1-4cef-4b6a-913b-cb0908a766ad service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:14.610301 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1359/2726] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:41:14.680347 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0559d2-fe16-422c-9112-cd17537f619f None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:14.680777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0559d2-fe16-422c-9112-cd17537f619f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:14.681122 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0559d2-fe16-422c-9112-cd17537f619f None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:14.682578 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1368/2727] 10.222.0.39 () {54 vars in 921 bytes} [Mon Aug 8 02:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:41:14.687427 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc46a8-1f24-4723-8eed-a8066293d396 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:14.687708 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc46a8-1f24-4723-8eed-a8066293d396 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:14.688024 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc46a8-1f24-4723-8eed-a8066293d396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:14.709589 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a6cc46a8-1f24-4723-8eed-a8066293d396 None None] MFA Rules not processed for user `78506fea1c024b169a4b7ee86116658c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:14.710813 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a6cc46a8-1f24-4723-8eed-a8066293d396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:14.717374 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1360/2728] 10.222.0.39 () {58 vars in 1018 bytes} [Mon Aug 8 02:41:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:41:14.729964 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0559d2-fe16-422c-9112-cd17537f619f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:14.740900 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0559d2-fe16-422c-9112-cd17537f619f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:14.744473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a02205-6c7f-4ff1-902d-13ff7a626ec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:14.745241 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a02205-6c7f-4ff1-902d-13ff7a626ec8 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:14.747878 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a02205-6c7f-4ff1-902d-13ff7a626ec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:14.748159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a02205-6c7f-4ff1-902d-13ff7a626ec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:14.755594 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a02205-6c7f-4ff1-902d-13ff7a626ec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:14.757013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a02205-6c7f-4ff1-902d-13ff7a626ec8 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:14.765006 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1369/2729] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:41:14.857054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2451e-09f3-48b7-a238-9f0822ef108d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:14.857433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2451e-09f3-48b7-a238-9f0822ef108d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:14.857777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2451e-09f3-48b7-a238-9f0822ef108d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:14.859183 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1361/2730] 10.222.0.39 () {54 vars in 921 bytes} [Mon Aug 8 02:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:41:14.864647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27e8ff-1cd5-41d6-a448-d01c29e00079 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:14.864928 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27e8ff-1cd5-41d6-a448-d01c29e00079 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:14.865198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27e8ff-1cd5-41d6-a448-d01c29e00079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:14.885094 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-0c27e8ff-1cd5-41d6-a448-d01c29e00079 None None] MFA Rules not processed for user `c4238566a9c248ffbfa314f47f02670c`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:14.886326 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0c27e8ff-1cd5-41d6-a448-d01c29e00079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:14.897230 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1370/2731] 10.222.0.39 () {58 vars in 1018 bytes} [Mon Aug 8 02:41:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:14.908325 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2451e-09f3-48b7-a238-9f0822ef108d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:14.919879 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2451e-09f3-48b7-a238-9f0822ef108d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:14.924114 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1178e2-5455-4f31-9458-8957be03dd77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:14.924731 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1178e2-5455-4f31-9458-8957be03dd77 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:14.925004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1178e2-5455-4f31-9458-8957be03dd77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:14.925267 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1178e2-5455-4f31-9458-8957be03dd77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:14.943184 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1178e2-5455-4f31-9458-8957be03dd77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:14.944467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1178e2-5455-4f31-9458-8957be03dd77 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:14.956392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1362/2732] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:41:17.462620 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79866485-8d34-4fb3-9835-4daab7a756a2 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:17.463151 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79866485-8d34-4fb3-9835-4daab7a756a2 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:17.463151 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79866485-8d34-4fb3-9835-4daab7a756a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:17.484430 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-79866485-8d34-4fb3-9835-4daab7a756a2 None None] MFA Rules not processed for user `b88ccd3e5c4c48d3a27f4eaa7874e28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:17.486208 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-79866485-8d34-4fb3-9835-4daab7a756a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:17.495209 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1371/2733] 10.222.0.39 () {58 vars in 1009 bytes} [Mon Aug 8 02:41:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:17.508379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1178e2-5455-4f31-9458-8957be03dd77 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:17.520029 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1178e2-5455-4f31-9458-8957be03dd77 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:17.525046 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef613eb-437f-4a90-b1b6-fda9942d5e5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac60b4d621234225830c94edfac400e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:17.525665 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef613eb-437f-4a90-b1b6-fda9942d5e5c service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:17.525943 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef613eb-437f-4a90-b1b6-fda9942d5e5c service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:17.526214 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef613eb-437f-4a90-b1b6-fda9942d5e5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:17.536205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef613eb-437f-4a90-b1b6-fda9942d5e5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:17.537777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef613eb-437f-4a90-b1b6-fda9942d5e5c service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:17.545959 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1363/2734] 10.222.0.39 () {60 vars in 1384 bytes} [Mon Aug 8 02:41:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:41:27.478254 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=21 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:27.542586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=46 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:32.827620 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79866485-8d34-4fb3-9835-4daab7a756a2 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:32.829154 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-79866485-8d34-4fb3-9835-4daab7a756a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:32.845364 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79866485-8d34-4fb3-9835-4daab7a756a2 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:32.851558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20002e3d-5cba-48c2-84f3-812212e629b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:32.852147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20002e3d-5cba-48c2-84f3-812212e629b8 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:32.852427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20002e3d-5cba-48c2-84f3-812212e629b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:32.852700 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20002e3d-5cba-48c2-84f3-812212e629b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:32.854447 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-20002e3d-5cba-48c2-84f3-812212e629b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:32.866797 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20002e3d-5cba-48c2-84f3-812212e629b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:32.868369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20002e3d-5cba-48c2-84f3-812212e629b8 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:32.878830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1372/2735] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:41:35.045232 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09e12d-184c-4f7b-b3d1-85529d77e4dd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:35.045741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09e12d-184c-4f7b-b3d1-85529d77e4dd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:35.045741 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09e12d-184c-4f7b-b3d1-85529d77e4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:35.065062 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2c09e12d-184c-4f7b-b3d1-85529d77e4dd None None] MFA Rules not processed for user `b88ccd3e5c4c48d3a27f4eaa7874e28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:35.067193 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2c09e12d-184c-4f7b-b3d1-85529d77e4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:35.074804 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1364/2736] 10.222.0.39 () {58 vars in 967 bytes} [Mon Aug 8 02:41:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:35.088262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20002e3d-5cba-48c2-84f3-812212e629b8 service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:35.102427 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20002e3d-5cba-48c2-84f3-812212e629b8 service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:35.108158 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2d78e-4be0-4b4d-a2d2-af29652badaf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac60b4d621234225830c94edfac400e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:35.109038 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2d78e-4be0-4b4d-a2d2-af29652badaf service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:35.109333 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2d78e-4be0-4b4d-a2d2-af29652badaf service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:35.109597 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2d78e-4be0-4b4d-a2d2-af29652badaf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:35.123325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2d78e-4be0-4b4d-a2d2-af29652badaf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:35.124916 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2d78e-4be0-4b4d-a2d2-af29652badaf service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:35.134885 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1373/2737] 10.222.0.39 () {60 vars in 1384 bytes} [Mon Aug 8 02:41:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:41:42.919336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09e12d-184c-4f7b-b3d1-85529d77e4dd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:42.928706 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09e12d-184c-4f7b-b3d1-85529d77e4dd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:42.932294 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655c383-767f-434b-8499-bf8f80fc0e55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:42.932881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655c383-767f-434b-8499-bf8f80fc0e55 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:42.933162 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655c383-767f-434b-8499-bf8f80fc0e55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:42.933433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655c383-767f-434b-8499-bf8f80fc0e55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:42.935156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f655c383-767f-434b-8499-bf8f80fc0e55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:42.944923 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f655c383-767f-434b-8499-bf8f80fc0e55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:42.946425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f655c383-767f-434b-8499-bf8f80fc0e55 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:42.954478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1365/2738] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:41:45.130689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=12 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:47.409562 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2d78e-4be0-4b4d-a2d2-af29652badaf service swift] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:47.411122 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5fa2d78e-4be0-4b4d-a2d2-af29652badaf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:47.427827 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2d78e-4be0-4b4d-a2d2-af29652badaf service swift] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:47.434608 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740f7f7-5716-4660-bba0-43591bfeb611 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:47.434608 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740f7f7-5716-4660-bba0-43591bfeb611 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:47.434608 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740f7f7-5716-4660-bba0-43591bfeb611 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:47.434995 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740f7f7-5716-4660-bba0-43591bfeb611 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:47.436765 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f740f7f7-5716-4660-bba0-43591bfeb611 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:47.451175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740f7f7-5716-4660-bba0-43591bfeb611 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:47.452964 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740f7f7-5716-4660-bba0-43591bfeb611 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:47.464979 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1374/2739] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:41:52.950917 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=9 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:57.460173 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=6 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:42:46.615159 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655c383-767f-434b-8499-bf8f80fc0e55 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:46.629894 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655c383-767f-434b-8499-bf8f80fc0e55 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:46.635484 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131b78e-391d-468f-a14b-dbd3614b066a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:46.636329 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131b78e-391d-468f-a14b-dbd3614b066a service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:46.636770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131b78e-391d-468f-a14b-dbd3614b066a service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:46.637210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131b78e-391d-468f-a14b-dbd3614b066a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:46.639128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7131b78e-391d-468f-a14b-dbd3614b066a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:46.653148 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131b78e-391d-468f-a14b-dbd3614b066a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:46.655061 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131b78e-391d-468f-a14b-dbd3614b066a service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:46.668867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1366/2740] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:42:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:42:51.504844 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740f7f7-5716-4660-bba0-43591bfeb611 service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:51.514435 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740f7f7-5716-4660-bba0-43591bfeb611 service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:51.518695 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0490b1-d96b-4c2f-9009-98ae6489b1d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:51.519279 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0490b1-d96b-4c2f-9009-98ae6489b1d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:51.519554 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0490b1-d96b-4c2f-9009-98ae6489b1d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:51.519821 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0490b1-d96b-4c2f-9009-98ae6489b1d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:51.521504 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7c0490b1-d96b-4c2f-9009-98ae6489b1d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:51.583088 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0490b1-d96b-4c2f-9009-98ae6489b1d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:51.584507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0490b1-d96b-4c2f-9009-98ae6489b1d4 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:51.594037 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1375/2741] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:42:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:42:54.645736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131b78e-391d-468f-a14b-dbd3614b066a service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:54.647966 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7131b78e-391d-468f-a14b-dbd3614b066a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:54.666898 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131b78e-391d-468f-a14b-dbd3614b066a service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:54.672275 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8e5e1-219c-405a-a763-cbe6abd00825 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f62745b92a864480a9438de581e1d05e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:54.672891 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8e5e1-219c-405a-a763-cbe6abd00825 service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:54.673221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8e5e1-219c-405a-a763-cbe6abd00825 service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:54.673496 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8e5e1-219c-405a-a763-cbe6abd00825 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:54.675465 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-41e8e5e1-219c-405a-a763-cbe6abd00825 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:54.688940 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8e5e1-219c-405a-a763-cbe6abd00825 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:54.691172 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8e5e1-219c-405a-a763-cbe6abd00825 service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:54.703830 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1367/2742] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:42:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:42:54.705244 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0490b1-d96b-4c2f-9009-98ae6489b1d4 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:54.707266 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7c0490b1-d96b-4c2f-9009-98ae6489b1d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:54.732759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0490b1-d96b-4c2f-9009-98ae6489b1d4 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:54.738098 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0b467-d88a-4bf1-8073-fedc85a33e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:54.738775 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0b467-d88a-4bf1-8073-fedc85a33e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:54.739052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0b467-d88a-4bf1-8073-fedc85a33e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:54.739327 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0b467-d88a-4bf1-8073-fedc85a33e2b admin admin] PATH_INFO: `/v3/users/5de4e28bfee14bae93c7f7a8f3f098fc` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:54.740503 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0b467-d88a-4bf1-8073-fedc85a33e2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5de4e28bfee14bae93c7f7a8f3f098fc)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:54.742543 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0b467-d88a-4bf1-8073-fedc85a33e2b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:54.790221 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-49a0b467-d88a-4bf1-8073-fedc85a33e2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5de4e28bfee14bae93c7f7a8f3f098fc'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:54.797045 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-49a0b467-d88a-4bf1-8073-fedc85a33e2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5de4e28bfee14bae93c7f7a8f3f098fc'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:54.805056 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-49a0b467-d88a-4bf1-8073-fedc85a33e2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5de4e28bfee14bae93c7f7a8f3f098fc'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:54.805675 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-49a0b467-d88a-4bf1-8073-fedc85a33e2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5de4e28bfee14bae93c7f7a8f3f098fc'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:54.806329 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-49a0b467-d88a-4bf1-8073-fedc85a33e2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5de4e28bfee14bae93c7f7a8f3f098fc'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:54.825514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1376/2743] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:42:54 2022] DELETE /identity/v3/users/5de4e28bfee14bae93c7f7a8f3f098fc => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:42:55.000560 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8e5e1-219c-405a-a763-cbe6abd00825 service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:55.011994 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8e5e1-219c-405a-a763-cbe6abd00825 service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:55.075417 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21fb51-7d18-474a-9520-38d1e55ecd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:55.076035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21fb51-7d18-474a-9520-38d1e55ecd0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:55.076317 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21fb51-7d18-474a-9520-38d1e55ecd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:55.076590 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21fb51-7d18-474a-9520-38d1e55ecd0f admin admin] PATH_INFO: `/v3/projects/6dcedd52439b47689047813e0566ba78` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:55.077707 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21fb51-7d18-474a-9520-38d1e55ecd0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dcedd52439b47689047813e0566ba78)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:55.079634 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21fb51-7d18-474a-9520-38d1e55ecd0f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:55.124841 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fb21fb51-7d18-474a-9520-38d1e55ecd0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dcedd52439b47689047813e0566ba78', 'request_id': 'req-fb21fb51-7d18-474a-9520-38d1e55ecd0f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:55.130884 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fb21fb51-7d18-474a-9520-38d1e55ecd0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dcedd52439b47689047813e0566ba78', 'request_id': 'req-fb21fb51-7d18-474a-9520-38d1e55ecd0f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:55.138106 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fb21fb51-7d18-474a-9520-38d1e55ecd0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dcedd52439b47689047813e0566ba78', 'request_id': 'req-fb21fb51-7d18-474a-9520-38d1e55ecd0f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:55.148952 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fb21fb51-7d18-474a-9520-38d1e55ecd0f admin admin] The token cache is being invalidate because project 6dcedd52439b47689047813e0566ba78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:42:55.149241 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fb21fb51-7d18-474a-9520-38d1e55ecd0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:55.149872 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fb21fb51-7d18-474a-9520-38d1e55ecd0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:55.151359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1368/2744] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:42:54 2022] DELETE /identity/v3/projects/6dcedd52439b47689047813e0566ba78 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:42:55.156387 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ae865-c220-4e2a-9650-4f8a97bf8610 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:55.156678 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ae865-c220-4e2a-9650-4f8a97bf8610 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:55.156956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ae865-c220-4e2a-9650-4f8a97bf8610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:55.178493 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-071ae865-c220-4e2a-9650-4f8a97bf8610 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:42:55.179641 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-071ae865-c220-4e2a-9650-4f8a97bf8610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:55.237621 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1377/2745] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:42:55.246133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c4ae9-ef1d-40a0-bf44-38ee739a73af None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:55.246457 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c4ae9-ef1d-40a0-bf44-38ee739a73af None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:55.246730 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c4ae9-ef1d-40a0-bf44-38ee739a73af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:55.265678 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b43c4ae9-ef1d-40a0-bf44-38ee739a73af None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:42:55.266908 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b43c4ae9-ef1d-40a0-bf44-38ee739a73af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:55.274660 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1369/2746] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:42:55.278693 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ae865-c220-4e2a-9650-4f8a97bf8610 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:55.288792 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ae865-c220-4e2a-9650-4f8a97bf8610 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:55.293563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60ef51-a704-44be-8669-e1f79112c29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:55.294144 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60ef51-a704-44be-8669-e1f79112c29f admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:55.294445 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60ef51-a704-44be-8669-e1f79112c29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:55.294714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60ef51-a704-44be-8669-e1f79112c29f admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:55.295346 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60ef51-a704-44be-8669-e1f79112c29f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:42:55.295647 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60ef51-a704-44be-8669-e1f79112c29f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:55.297225 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60ef51-a704-44be-8669-e1f79112c29f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:55.298666 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:42:55.314826 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1378/2747] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:42:55.316087 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c4ae9-ef1d-40a0-bf44-38ee739a73af None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:55.325343 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c4ae9-ef1d-40a0-bf44-38ee739a73af None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:55.329928 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f301d1-b1b7-4b83-bad9-b288d8d9f397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:55.330538 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f301d1-b1b7-4b83-bad9-b288d8d9f397 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:55.330814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f301d1-b1b7-4b83-bad9-b288d8d9f397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:55.331077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f301d1-b1b7-4b83-bad9-b288d8d9f397 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:55.331694 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f301d1-b1b7-4b83-bad9-b288d8d9f397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:55.333569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f301d1-b1b7-4b83-bad9-b288d8d9f397 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:55.361135 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1370/2748] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:42:55 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:42:55.365770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60ef51-a704-44be-8669-e1f79112c29f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:55.375671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60ef51-a704-44be-8669-e1f79112c29f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:55.432282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be848241-62f0-4cf6-85a5-b8524b273fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:55.432803 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be848241-62f0-4cf6-85a5-b8524b273fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:55.433076 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be848241-62f0-4cf6-85a5-b8524b273fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:55.433343 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be848241-62f0-4cf6-85a5-b8524b273fb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:55.433973 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be848241-62f0-4cf6-85a5-b8524b273fb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:55.435890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be848241-62f0-4cf6-85a5-b8524b273fb1 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:55.458793 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1379/2749] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:42:55 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:42:55.462924 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f301d1-b1b7-4b83-bad9-b288d8d9f397 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:55.473840 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f301d1-b1b7-4b83-bad9-b288d8d9f397 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:55.478504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6850e-1824-4d30-84e0-58e45823e563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:55.479090 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6850e-1824-4d30-84e0-58e45823e563 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:55.479370 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6850e-1824-4d30-84e0-58e45823e563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:55.479648 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6850e-1824-4d30-84e0-58e45823e563 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:55.480306 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b6850e-1824-4d30-84e0-58e45823e563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:55.481889 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b6850e-1824-4d30-84e0-58e45823e563 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:55.492569 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1371/2750] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:42:55.496466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be848241-62f0-4cf6-85a5-b8524b273fb1 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:55.506442 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be848241-62f0-4cf6-85a5-b8524b273fb1 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:55.511119 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6d300-05b3-480f-b6fb-8f7e23daaca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:55.511702 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6d300-05b3-480f-b6fb-8f7e23daaca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:55.511978 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6d300-05b3-480f-b6fb-8f7e23daaca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:55.512242 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6d300-05b3-480f-b6fb-8f7e23daaca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:55.512893 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6d300-05b3-480f-b6fb-8f7e23daaca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:55.514480 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6d300-05b3-480f-b6fb-8f7e23daaca9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:55.525411 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1380/2751] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:42:55.529512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6850e-1824-4d30-84e0-58e45823e563 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:55.540088 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6850e-1824-4d30-84e0-58e45823e563 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:55.544683 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab34847-4756-4cc2-838f-d75339da172d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:55.545322 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab34847-4756-4cc2-838f-d75339da172d admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:55.545601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab34847-4756-4cc2-838f-d75339da172d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:55.545873 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab34847-4756-4cc2-838f-d75339da172d admin admin] PATH_INFO: `/v3/projects/369eb0d4c90a439da8b1983e9dfea183/users/502200cb5466455491e1d4e4ba501b37/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:55.560451 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab34847-4756-4cc2-838f-d75339da172d admin admin] RBAC: Authorizing `identity:create_grant(project_id=369eb0d4c90a439da8b1983e9dfea183, user_id=502200cb5466455491e1d4e4ba501b37, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:55.563125 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab34847-4756-4cc2-838f-d75339da172d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:55.572465 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:42:55.577330 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1372/2752] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:42:55 2022] PUT /identity/v3/projects/369eb0d4c90a439da8b1983e9dfea183/users/502200cb5466455491e1d4e4ba501b37/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:42:55.589430 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900e1f5-ae34-467b-9c03-08e7b51aa93e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:55.589718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900e1f5-ae34-467b-9c03-08e7b51aa93e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:55.589994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900e1f5-ae34-467b-9c03-08e7b51aa93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:55.617729 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-2900e1f5-ae34-467b-9c03-08e7b51aa93e None None] MFA Rules not processed for user `502200cb5466455491e1d4e4ba501b37`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:42:55.618969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2900e1f5-ae34-467b-9c03-08e7b51aa93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:55.683162 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1381/2753] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:42:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:42:55.694904 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab34847-4756-4cc2-838f-d75339da172d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:55.696620 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1ab34847-4756-4cc2-838f-d75339da172d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:55.757384 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab34847-4756-4cc2-838f-d75339da172d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:55.761050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06ed6a-c7cf-4dc1-ba48-3732dbbd91fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:55.761642 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06ed6a-c7cf-4dc1-ba48-3732dbbd91fe service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:55.761926 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06ed6a-c7cf-4dc1-ba48-3732dbbd91fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:55.762199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06ed6a-c7cf-4dc1-ba48-3732dbbd91fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:55.770473 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06ed6a-c7cf-4dc1-ba48-3732dbbd91fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:55.771903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06ed6a-c7cf-4dc1-ba48-3732dbbd91fe service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:55.780766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1373/2754] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:42:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:04.091383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900e1f5-ae34-467b-9c03-08e7b51aa93e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:04.151796 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900e1f5-ae34-467b-9c03-08e7b51aa93e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:04.156171 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd408962-9764-4963-b93b-16d5a52447b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:04.156781 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd408962-9764-4963-b93b-16d5a52447b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:04.157053 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd408962-9764-4963-b93b-16d5a52447b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:04.157319 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd408962-9764-4963-b93b-16d5a52447b7 admin admin] PATH_INFO: `/v3/users/502200cb5466455491e1d4e4ba501b37` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:04.158469 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd408962-9764-4963-b93b-16d5a52447b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=502200cb5466455491e1d4e4ba501b37)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:04.160855 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd408962-9764-4963-b93b-16d5a52447b7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:04.163042 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:04.209008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-fd408962-9764-4963-b93b-16d5a52447b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '502200cb5466455491e1d4e4ba501b37'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:04.216083 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-fd408962-9764-4963-b93b-16d5a52447b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '502200cb5466455491e1d4e4ba501b37'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:04.223583 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-fd408962-9764-4963-b93b-16d5a52447b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '502200cb5466455491e1d4e4ba501b37'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:04.224469 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-fd408962-9764-4963-b93b-16d5a52447b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '502200cb5466455491e1d4e4ba501b37'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:04.225242 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-fd408962-9764-4963-b93b-16d5a52447b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '502200cb5466455491e1d4e4ba501b37'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:04.242913 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1382/2755] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:43:04 2022] DELETE /identity/v3/users/502200cb5466455491e1d4e4ba501b37 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:04.247937 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8708fe-23a5-4ce5-ab22-37dff81c238b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:04.248480 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8708fe-23a5-4ce5-ab22-37dff81c238b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:04.248823 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8708fe-23a5-4ce5-ab22-37dff81c238b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:04.267246 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ff8708fe-23a5-4ce5-ab22-37dff81c238b None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:04.268428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ff8708fe-23a5-4ce5-ab22-37dff81c238b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:04.326513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1374/2756] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:04.336186 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd408962-9764-4963-b93b-16d5a52447b7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:04.337674 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-fd408962-9764-4963-b93b-16d5a52447b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:04.394038 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd408962-9764-4963-b93b-16d5a52447b7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:04.397697 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4ec0d-ece7-419a-b9b8-52a43a0c0f6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:04.398348 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4ec0d-ece7-419a-b9b8-52a43a0c0f6f service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:04.398632 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4ec0d-ece7-419a-b9b8-52a43a0c0f6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:04.398905 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4ec0d-ece7-419a-b9b8-52a43a0c0f6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:04.407920 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4ec0d-ece7-419a-b9b8-52a43a0c0f6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:04.409327 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4ec0d-ece7-419a-b9b8-52a43a0c0f6f service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:04.419516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1383/2757] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:04.686318 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8708fe-23a5-4ce5-ab22-37dff81c238b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:04.687795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ff8708fe-23a5-4ce5-ab22-37dff81c238b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:04.701902 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8708fe-23a5-4ce5-ab22-37dff81c238b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:04.708068 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f03677-ba6c-4a19-9a82-9a9e1a1d0dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:04.709048 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f03677-ba6c-4a19-9a82-9a9e1a1d0dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:04.709337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f03677-ba6c-4a19-9a82-9a9e1a1d0dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:04.709581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f03677-ba6c-4a19-9a82-9a9e1a1d0dca admin admin] PATH_INFO: `/v3/projects/369eb0d4c90a439da8b1983e9dfea183` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:04.710831 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f03677-ba6c-4a19-9a82-9a9e1a1d0dca admin admin] RBAC: Authorizing `identity:delete_project(project_id=369eb0d4c90a439da8b1983e9dfea183)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:04.713102 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f03677-ba6c-4a19-9a82-9a9e1a1d0dca admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:04.760429 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e6f03677-ba6c-4a19-9a82-9a9e1a1d0dca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '369eb0d4c90a439da8b1983e9dfea183', 'request_id': 'req-e6f03677-ba6c-4a19-9a82-9a9e1a1d0dca'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:04.765854 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e6f03677-ba6c-4a19-9a82-9a9e1a1d0dca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '369eb0d4c90a439da8b1983e9dfea183', 'request_id': 'req-e6f03677-ba6c-4a19-9a82-9a9e1a1d0dca'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:04.773512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e6f03677-ba6c-4a19-9a82-9a9e1a1d0dca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '369eb0d4c90a439da8b1983e9dfea183', 'request_id': 'req-e6f03677-ba6c-4a19-9a82-9a9e1a1d0dca'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:04.785067 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e6f03677-ba6c-4a19-9a82-9a9e1a1d0dca admin admin] The token cache is being invalidate because project 369eb0d4c90a439da8b1983e9dfea183 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:43:04.785357 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e6f03677-ba6c-4a19-9a82-9a9e1a1d0dca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:04.785982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e6f03677-ba6c-4a19-9a82-9a9e1a1d0dca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:04.787465 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1375/2758] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:43:04 2022] DELETE /identity/v3/projects/369eb0d4c90a439da8b1983e9dfea183 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:04.792549 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa37539-8af6-4576-8847-d51151d760a7 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:04.792913 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa37539-8af6-4576-8847-d51151d760a7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:04.793188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa37539-8af6-4576-8847-d51151d760a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:04.812137 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-afa37539-8af6-4576-8847-d51151d760a7 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:04.813331 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-afa37539-8af6-4576-8847-d51151d760a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:04.872020 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1384/2759] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:04.880383 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d67100-aec3-4313-890e-3fb866af7038 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:04.880778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d67100-aec3-4313-890e-3fb866af7038 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:04.881055 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d67100-aec3-4313-890e-3fb866af7038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:04.901285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e4d67100-aec3-4313-890e-3fb866af7038 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:04.902506 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e4d67100-aec3-4313-890e-3fb866af7038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:04.910772 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1376/2760] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:04.914919 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa37539-8af6-4576-8847-d51151d760a7 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:04.924309 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa37539-8af6-4576-8847-d51151d760a7 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:04.928856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da202b-f022-413b-9eae-ec1d0d851848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:04.929431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da202b-f022-413b-9eae-ec1d0d851848 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:04.929705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da202b-f022-413b-9eae-ec1d0d851848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:04.929971 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da202b-f022-413b-9eae-ec1d0d851848 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:04.930650 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da202b-f022-413b-9eae-ec1d0d851848 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:43:04.930950 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da202b-f022-413b-9eae-ec1d0d851848 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:04.932516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da202b-f022-413b-9eae-ec1d0d851848 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:04.944383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1385/2761] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:43:04.948357 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d67100-aec3-4313-890e-3fb866af7038 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:04.958018 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d67100-aec3-4313-890e-3fb866af7038 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:04.962601 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1fab6-ec71-4b33-b72a-9b8d691d35a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:04.963175 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1fab6-ec71-4b33-b72a-9b8d691d35a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:04.963447 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1fab6-ec71-4b33-b72a-9b8d691d35a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:04.963719 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1fab6-ec71-4b33-b72a-9b8d691d35a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:04.964336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f1fab6-ec71-4b33-b72a-9b8d691d35a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:04.966201 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f1fab6-ec71-4b33-b72a-9b8d691d35a7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:04.986950 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:04.996160 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1377/2762] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:43:04 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:43:05.000242 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da202b-f022-413b-9eae-ec1d0d851848 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:05.010486 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da202b-f022-413b-9eae-ec1d0d851848 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:05.018060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:05.068674 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce8d13-fd51-4895-b789-3db31e2d7e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:05.069275 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce8d13-fd51-4895-b789-3db31e2d7e69 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:05.069555 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce8d13-fd51-4895-b789-3db31e2d7e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:05.069856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce8d13-fd51-4895-b789-3db31e2d7e69 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:05.070545 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce8d13-fd51-4895-b789-3db31e2d7e69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:05.072474 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce8d13-fd51-4895-b789-3db31e2d7e69 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:05.093004 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1386/2763] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:43:04 2022] POST /identity/v3/users => generated 278 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:43:05.097150 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1fab6-ec71-4b33-b72a-9b8d691d35a7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:05.107419 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1fab6-ec71-4b33-b72a-9b8d691d35a7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:05.112059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbadd30-0921-4240-989d-672887eacce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:05.112647 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbadd30-0921-4240-989d-672887eacce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:05.112924 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbadd30-0921-4240-989d-672887eacce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:05.113192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbadd30-0921-4240-989d-672887eacce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:05.113822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbadd30-0921-4240-989d-672887eacce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:05.115454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbadd30-0921-4240-989d-672887eacce9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:05.125257 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1378/2764] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:05.129336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce8d13-fd51-4895-b789-3db31e2d7e69 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:05.139095 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce8d13-fd51-4895-b789-3db31e2d7e69 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:05.143688 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f0e1d-d18f-470f-9b63-9228dd6904f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:05.144836 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f0e1d-d18f-470f-9b63-9228dd6904f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:05.145112 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f0e1d-d18f-470f-9b63-9228dd6904f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:05.145380 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f0e1d-d18f-470f-9b63-9228dd6904f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:05.146037 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f0e1d-d18f-470f-9b63-9228dd6904f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:05.147635 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f0e1d-d18f-470f-9b63-9228dd6904f8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:05.158938 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1387/2765] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:05.162686 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbadd30-0921-4240-989d-672887eacce9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:05.172300 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbadd30-0921-4240-989d-672887eacce9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:05.176921 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74cc45-4d06-434a-a2a3-33cc3b848f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:05.177545 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74cc45-4d06-434a-a2a3-33cc3b848f00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:05.177821 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74cc45-4d06-434a-a2a3-33cc3b848f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:05.178086 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74cc45-4d06-434a-a2a3-33cc3b848f00 admin admin] PATH_INFO: `/v3/projects/7b30bdbdae444fa792a19b11e405b37c/users/57fb79b05a6f46bf9742a355d406b958/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:05.192679 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74cc45-4d06-434a-a2a3-33cc3b848f00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b30bdbdae444fa792a19b11e405b37c, user_id=57fb79b05a6f46bf9742a355d406b958, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:05.195347 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74cc45-4d06-434a-a2a3-33cc3b848f00 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:05.208767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1379/2766] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:43:05 2022] PUT /identity/v3/projects/7b30bdbdae444fa792a19b11e405b37c/users/57fb79b05a6f46bf9742a355d406b958/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:05.221583 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8a81d-70f0-45a4-9011-d4787bc15702 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:05.221897 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8a81d-70f0-45a4-9011-d4787bc15702 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:05.222180 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8a81d-70f0-45a4-9011-d4787bc15702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:05.249778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-35a8a81d-70f0-45a4-9011-d4787bc15702 None None] MFA Rules not processed for user `57fb79b05a6f46bf9742a355d406b958`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:05.251035 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-35a8a81d-70f0-45a4-9011-d4787bc15702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:05.313869 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1388/2767] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:05.319484 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74cc45-4d06-434a-a2a3-33cc3b848f00 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:05.379552 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74cc45-4d06-434a-a2a3-33cc3b848f00 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:05.384016 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb6494-459a-44be-8ccd-c22cde007f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:05.384603 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb6494-459a-44be-8ccd-c22cde007f55 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:05.384876 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb6494-459a-44be-8ccd-c22cde007f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:05.385145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb6494-459a-44be-8ccd-c22cde007f55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:05.385746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb6494-459a-44be-8ccd-c22cde007f55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:05.387688 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb6494-459a-44be-8ccd-c22cde007f55 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:05.413210 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1380/2768] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:43:05 2022] POST /identity/v3/projects => generated 357 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:43:05.417543 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8a81d-70f0-45a4-9011-d4787bc15702 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:05.428954 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8a81d-70f0-45a4-9011-d4787bc15702 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:05.483896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9819b-273e-4617-951d-af2cf6275ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:05.484496 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9819b-273e-4617-951d-af2cf6275ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:05.484780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9819b-273e-4617-951d-af2cf6275ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:05.485056 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9819b-273e-4617-951d-af2cf6275ed3 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:05.485686 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9819b-273e-4617-951d-af2cf6275ed3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:05.488115 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9819b-273e-4617-951d-af2cf6275ed3 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:05.509100 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1389/2769] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:43:05 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:43:05.513040 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb6494-459a-44be-8ccd-c22cde007f55 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:05.523368 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb6494-459a-44be-8ccd-c22cde007f55 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:05.528068 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e482d83-f590-4966-a4bf-a33f3884a46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:05.528656 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e482d83-f590-4966-a4bf-a33f3884a46d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:05.528934 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e482d83-f590-4966-a4bf-a33f3884a46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:05.529204 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e482d83-f590-4966-a4bf-a33f3884a46d admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:05.529838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e482d83-f590-4966-a4bf-a33f3884a46d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:05.531463 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e482d83-f590-4966-a4bf-a33f3884a46d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:05.542632 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1381/2770] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:05.546802 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9819b-273e-4617-951d-af2cf6275ed3 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:05.557134 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9819b-273e-4617-951d-af2cf6275ed3 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:05.561664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ac37b-e6ae-4edd-ad95-8caa2e873187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:05.562335 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ac37b-e6ae-4edd-ad95-8caa2e873187 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:05.562618 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ac37b-e6ae-4edd-ad95-8caa2e873187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:05.562890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ac37b-e6ae-4edd-ad95-8caa2e873187 admin admin] PATH_INFO: `/v3/projects/0e723a7bb22e493a8f88ecf50b558597/users/c82bc5cbca8c4df98061a2dada5e618d/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:05.566760 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:05.579648 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ac37b-e6ae-4edd-ad95-8caa2e873187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e723a7bb22e493a8f88ecf50b558597, user_id=c82bc5cbca8c4df98061a2dada5e618d, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:05.582307 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ac37b-e6ae-4edd-ad95-8caa2e873187 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:05.594822 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1390/2771] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:43:05 2022] PUT /identity/v3/projects/0e723a7bb22e493a8f88ecf50b558597/users/c82bc5cbca8c4df98061a2dada5e618d/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:05.599297 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e482d83-f590-4966-a4bf-a33f3884a46d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:05.609423 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e482d83-f590-4966-a4bf-a33f3884a46d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:05.667063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900668b-ffd9-49e4-92e9-41f731de4bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:05.667580 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900668b-ffd9-49e4-92e9-41f731de4bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:05.667857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900668b-ffd9-49e4-92e9-41f731de4bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:05.668123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900668b-ffd9-49e4-92e9-41f731de4bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:05.668746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900668b-ffd9-49e4-92e9-41f731de4bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:05.670387 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900668b-ffd9-49e4-92e9-41f731de4bc7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:05.680522 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1382/2772] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:05.684589 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ac37b-e6ae-4edd-ad95-8caa2e873187 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:05.695418 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ac37b-e6ae-4edd-ad95-8caa2e873187 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:05.700107 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f12a55-7aa9-4b43-8f99-a92dd374a080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:05.700737 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f12a55-7aa9-4b43-8f99-a92dd374a080 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:05.701014 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f12a55-7aa9-4b43-8f99-a92dd374a080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:05.701284 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f12a55-7aa9-4b43-8f99-a92dd374a080 admin admin] PATH_INFO: `/v3/projects/0e723a7bb22e493a8f88ecf50b558597/users/c82bc5cbca8c4df98061a2dada5e618d/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:05.703270 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f12a55-7aa9-4b43-8f99-a92dd374a080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e723a7bb22e493a8f88ecf50b558597, user_id=c82bc5cbca8c4df98061a2dada5e618d, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:05.705881 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f12a55-7aa9-4b43-8f99-a92dd374a080 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:05.715876 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1391/2773] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:43:05 2022] PUT /identity/v3/projects/0e723a7bb22e493a8f88ecf50b558597/users/c82bc5cbca8c4df98061a2dada5e618d/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:05.726300 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ae726-ce88-4d3c-bbe5-ef14da71e91a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:05.726593 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ae726-ce88-4d3c-bbe5-ef14da71e91a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:05.726872 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ae726-ce88-4d3c-bbe5-ef14da71e91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:05.754168 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b69ae726-ce88-4d3c-bbe5-ef14da71e91a None None] MFA Rules not processed for user `c82bc5cbca8c4df98061a2dada5e618d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:05.755411 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b69ae726-ce88-4d3c-bbe5-ef14da71e91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:05.779796 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:05.829386 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1383/2774] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:05 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:05.840662 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f12a55-7aa9-4b43-8f99-a92dd374a080 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:05.842407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-45f12a55-7aa9-4b43-8f99-a92dd374a080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:05.898986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f12a55-7aa9-4b43-8f99-a92dd374a080 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:05.902625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920602f7-1b68-491a-bb81-2db5eb4f6f30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:05.903219 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920602f7-1b68-491a-bb81-2db5eb4f6f30 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:05.903498 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920602f7-1b68-491a-bb81-2db5eb4f6f30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:05.903769 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920602f7-1b68-491a-bb81-2db5eb4f6f30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:05.913106 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920602f7-1b68-491a-bb81-2db5eb4f6f30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:05.914602 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920602f7-1b68-491a-bb81-2db5eb4f6f30 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:05.924075 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1392/2775] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:05 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:05.997403 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ae726-ce88-4d3c-bbe5-ef14da71e91a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:06.007185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ae726-ce88-4d3c-bbe5-ef14da71e91a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:06.010790 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c0f70-dbdd-48b6-89cf-aa2e2b25fe5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:06.011373 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c0f70-dbdd-48b6-89cf-aa2e2b25fe5b service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:06.011643 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c0f70-dbdd-48b6-89cf-aa2e2b25fe5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:06.011909 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c0f70-dbdd-48b6-89cf-aa2e2b25fe5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:06.067032 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c0f70-dbdd-48b6-89cf-aa2e2b25fe5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:06.068465 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c0f70-dbdd-48b6-89cf-aa2e2b25fe5b service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:06.077134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1384/2776] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:15.919926 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=37,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:16.073174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=47 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:21.105248 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920602f7-1b68-491a-bb81-2db5eb4f6f30 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:21.166295 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920602f7-1b68-491a-bb81-2db5eb4f6f30 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:21.170779 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587723aa-b4ee-4173-8538-0941c8b7718b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:21.171406 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587723aa-b4ee-4173-8538-0941c8b7718b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:21.171664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587723aa-b4ee-4173-8538-0941c8b7718b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:21.171940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587723aa-b4ee-4173-8538-0941c8b7718b admin admin] PATH_INFO: `/v3/users/57fb79b05a6f46bf9742a355d406b958` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:21.173047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587723aa-b4ee-4173-8538-0941c8b7718b admin admin] RBAC: Authorizing `identity:delete_user(user_id=57fb79b05a6f46bf9742a355d406b958)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:21.175771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587723aa-b4ee-4173-8538-0941c8b7718b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:21.221094 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-587723aa-b4ee-4173-8538-0941c8b7718b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57fb79b05a6f46bf9742a355d406b958'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:21.227928 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-587723aa-b4ee-4173-8538-0941c8b7718b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57fb79b05a6f46bf9742a355d406b958'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:21.235095 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-587723aa-b4ee-4173-8538-0941c8b7718b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57fb79b05a6f46bf9742a355d406b958'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:21.235712 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-587723aa-b4ee-4173-8538-0941c8b7718b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57fb79b05a6f46bf9742a355d406b958'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:21.236318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-587723aa-b4ee-4173-8538-0941c8b7718b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57fb79b05a6f46bf9742a355d406b958'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:21.255413 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1393/2777] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:43:21 2022] DELETE /identity/v3/users/57fb79b05a6f46bf9742a355d406b958 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:21.259961 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9777bd5-987f-4e13-b3fe-0644b89748de None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:21.260470 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9777bd5-987f-4e13-b3fe-0644b89748de None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:21.260586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9777bd5-987f-4e13-b3fe-0644b89748de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:21.278929 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f9777bd5-987f-4e13-b3fe-0644b89748de None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:21.280122 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f9777bd5-987f-4e13-b3fe-0644b89748de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:21.337276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1385/2778] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:21.347999 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587723aa-b4ee-4173-8538-0941c8b7718b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:21.349448 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-587723aa-b4ee-4173-8538-0941c8b7718b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:21.404817 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587723aa-b4ee-4173-8538-0941c8b7718b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:21.408807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7821c-83a9-4bf2-8666-7ac8b5e6a03a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:21.409890 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7821c-83a9-4bf2-8666-7ac8b5e6a03a service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:21.410166 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7821c-83a9-4bf2-8666-7ac8b5e6a03a service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:21.410469 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7821c-83a9-4bf2-8666-7ac8b5e6a03a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:21.419985 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7821c-83a9-4bf2-8666-7ac8b5e6a03a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:21.421446 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7821c-83a9-4bf2-8666-7ac8b5e6a03a service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:21.431622 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1394/2779] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:21.693864 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9777bd5-987f-4e13-b3fe-0644b89748de None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:21.695436 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f9777bd5-987f-4e13-b3fe-0644b89748de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:21.709464 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9777bd5-987f-4e13-b3fe-0644b89748de None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:21.714800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36611305-98c2-4f69-ae27-a6e3992801a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:21.715699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36611305-98c2-4f69-ae27-a6e3992801a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:21.716031 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36611305-98c2-4f69-ae27-a6e3992801a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:21.716236 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36611305-98c2-4f69-ae27-a6e3992801a5 admin admin] PATH_INFO: `/v3/projects/7b30bdbdae444fa792a19b11e405b37c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:21.717416 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36611305-98c2-4f69-ae27-a6e3992801a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b30bdbdae444fa792a19b11e405b37c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:21.719402 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36611305-98c2-4f69-ae27-a6e3992801a5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:21.765162 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-36611305-98c2-4f69-ae27-a6e3992801a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b30bdbdae444fa792a19b11e405b37c', 'request_id': 'req-36611305-98c2-4f69-ae27-a6e3992801a5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:21.771080 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-36611305-98c2-4f69-ae27-a6e3992801a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b30bdbdae444fa792a19b11e405b37c', 'request_id': 'req-36611305-98c2-4f69-ae27-a6e3992801a5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:21.778829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-36611305-98c2-4f69-ae27-a6e3992801a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b30bdbdae444fa792a19b11e405b37c', 'request_id': 'req-36611305-98c2-4f69-ae27-a6e3992801a5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:21.790405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-36611305-98c2-4f69-ae27-a6e3992801a5 admin admin] The token cache is being invalidate because project 7b30bdbdae444fa792a19b11e405b37c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:43:21.790700 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-36611305-98c2-4f69-ae27-a6e3992801a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:21.791472 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-36611305-98c2-4f69-ae27-a6e3992801a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:21.792968 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1386/2780] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:43:21 2022] DELETE /identity/v3/projects/7b30bdbdae444fa792a19b11e405b37c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:21.796981 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7821c-83a9-4bf2-8666-7ac8b5e6a03a service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:21.806872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7821c-83a9-4bf2-8666-7ac8b5e6a03a service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:21.828838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:21.869566 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44e8e9-10fd-4d39-ac59-7d6e06d75b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:21.870108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44e8e9-10fd-4d39-ac59-7d6e06d75b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:21.870418 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44e8e9-10fd-4d39-ac59-7d6e06d75b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:21.870687 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44e8e9-10fd-4d39-ac59-7d6e06d75b62 admin admin] PATH_INFO: `/v3/users/c82bc5cbca8c4df98061a2dada5e618d` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:21.871787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44e8e9-10fd-4d39-ac59-7d6e06d75b62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c82bc5cbca8c4df98061a2dada5e618d)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:21.873667 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44e8e9-10fd-4d39-ac59-7d6e06d75b62 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:21.918674 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4d44e8e9-10fd-4d39-ac59-7d6e06d75b62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c82bc5cbca8c4df98061a2dada5e618d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:21.925562 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4d44e8e9-10fd-4d39-ac59-7d6e06d75b62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c82bc5cbca8c4df98061a2dada5e618d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:21.932870 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4d44e8e9-10fd-4d39-ac59-7d6e06d75b62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c82bc5cbca8c4df98061a2dada5e618d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:21.933502 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4d44e8e9-10fd-4d39-ac59-7d6e06d75b62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c82bc5cbca8c4df98061a2dada5e618d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:21.934103 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4d44e8e9-10fd-4d39-ac59-7d6e06d75b62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c82bc5cbca8c4df98061a2dada5e618d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:21.951236 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1395/2781] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:43:21 2022] DELETE /identity/v3/users/c82bc5cbca8c4df98061a2dada5e618d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:22.467050 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36611305-98c2-4f69-ae27-a6e3992801a5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:22.468591 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-36611305-98c2-4f69-ae27-a6e3992801a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:22.481815 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36611305-98c2-4f69-ae27-a6e3992801a5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:22.539432 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f991e35-4538-4834-8368-d0422bafd8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:22.540054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f991e35-4538-4834-8368-d0422bafd8d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:22.540332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f991e35-4538-4834-8368-d0422bafd8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:22.540600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f991e35-4538-4834-8368-d0422bafd8d9 admin admin] PATH_INFO: `/v3/projects/0e723a7bb22e493a8f88ecf50b558597` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:22.541701 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f991e35-4538-4834-8368-d0422bafd8d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e723a7bb22e493a8f88ecf50b558597)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:22.543629 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f991e35-4538-4834-8368-d0422bafd8d9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:22.577653 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:22.594506 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f991e35-4538-4834-8368-d0422bafd8d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e723a7bb22e493a8f88ecf50b558597', 'request_id': 'req-0f991e35-4538-4834-8368-d0422bafd8d9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:22.599872 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f991e35-4538-4834-8368-d0422bafd8d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e723a7bb22e493a8f88ecf50b558597', 'request_id': 'req-0f991e35-4538-4834-8368-d0422bafd8d9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:22.606783 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f991e35-4538-4834-8368-d0422bafd8d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e723a7bb22e493a8f88ecf50b558597', 'request_id': 'req-0f991e35-4538-4834-8368-d0422bafd8d9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:22.618814 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f991e35-4538-4834-8368-d0422bafd8d9 admin admin] The token cache is being invalidate because project 0e723a7bb22e493a8f88ecf50b558597 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:43:22.619104 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f991e35-4538-4834-8368-d0422bafd8d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:22.619799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f991e35-4538-4834-8368-d0422bafd8d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:22.621288 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1387/2782] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:43:22 2022] DELETE /identity/v3/projects/0e723a7bb22e493a8f88ecf50b558597 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:22.626218 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14690048-094c-4820-a69a-8ba956c7cfa7 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:22.626722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14690048-094c-4820-a69a-8ba956c7cfa7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:22.626827 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14690048-094c-4820-a69a-8ba956c7cfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:22.647568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-14690048-094c-4820-a69a-8ba956c7cfa7 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:22.648758 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-14690048-094c-4820-a69a-8ba956c7cfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:22.705534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1396/2783] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:43:22.713256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2efc6b6-9a8d-4516-9424-987ea238cc3f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:22.713537 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2efc6b6-9a8d-4516-9424-987ea238cc3f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:22.713807 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2efc6b6-9a8d-4516-9424-987ea238cc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:22.734624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a2efc6b6-9a8d-4516-9424-987ea238cc3f None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:22.736655 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a2efc6b6-9a8d-4516-9424-987ea238cc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:22.747215 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1388/2784] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:22.751593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14690048-094c-4820-a69a-8ba956c7cfa7 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:22.762134 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14690048-094c-4820-a69a-8ba956c7cfa7 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:22.766756 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef39c88-089e-4ee4-b4c6-21119d9a961b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:22.767349 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef39c88-089e-4ee4-b4c6-21119d9a961b admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:22.767626 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef39c88-089e-4ee4-b4c6-21119d9a961b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:22.767925 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef39c88-089e-4ee4-b4c6-21119d9a961b admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:22.768578 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef39c88-089e-4ee4-b4c6-21119d9a961b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:43:22.768883 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef39c88-089e-4ee4-b4c6-21119d9a961b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:22.770501 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef39c88-089e-4ee4-b4c6-21119d9a961b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:22.781714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1397/2785] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:43:22.786061 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2efc6b6-9a8d-4516-9424-987ea238cc3f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:22.795523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2efc6b6-9a8d-4516-9424-987ea238cc3f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:22.800658 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f31b95-fbe2-41d6-8060-189d30fdae03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:22.801244 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f31b95-fbe2-41d6-8060-189d30fdae03 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:22.801541 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f31b95-fbe2-41d6-8060-189d30fdae03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:22.801861 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f31b95-fbe2-41d6-8060-189d30fdae03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:22.802515 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f31b95-fbe2-41d6-8060-189d30fdae03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:22.804400 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f31b95-fbe2-41d6-8060-189d30fdae03 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:22.831510 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1389/2786] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:43:22 2022] POST /identity/v3/projects => generated 361 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:43:22.835698 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef39c88-089e-4ee4-b4c6-21119d9a961b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:22.845779 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef39c88-089e-4ee4-b4c6-21119d9a961b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:22.871837 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:22.902799 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59799bac-fd52-4aea-849c-73e80ed495cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:22.903389 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59799bac-fd52-4aea-849c-73e80ed495cc admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:22.903665 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59799bac-fd52-4aea-849c-73e80ed495cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:22.903934 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59799bac-fd52-4aea-849c-73e80ed495cc admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:22.904556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59799bac-fd52-4aea-849c-73e80ed495cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:22.906510 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59799bac-fd52-4aea-849c-73e80ed495cc admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:22.927569 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1398/2787] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:43:22 2022] POST /identity/v3/users => generated 281 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:43:22.931608 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f31b95-fbe2-41d6-8060-189d30fdae03 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:22.942226 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f31b95-fbe2-41d6-8060-189d30fdae03 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:22.947058 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a28b1-a5f6-488e-b2f9-ed7cab8e5e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:22.947643 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a28b1-a5f6-488e-b2f9-ed7cab8e5e7e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:22.947920 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a28b1-a5f6-488e-b2f9-ed7cab8e5e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:22.948188 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a28b1-a5f6-488e-b2f9-ed7cab8e5e7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:22.948823 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a28b1-a5f6-488e-b2f9-ed7cab8e5e7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:22.950432 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a28b1-a5f6-488e-b2f9-ed7cab8e5e7e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:22.960140 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1390/2788] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:22.963807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59799bac-fd52-4aea-849c-73e80ed495cc admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:22.974029 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59799bac-fd52-4aea-849c-73e80ed495cc admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:22.978668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aea54c-109f-48e3-9c14-f480064876ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:22.979243 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aea54c-109f-48e3-9c14-f480064876ba admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:22.979516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aea54c-109f-48e3-9c14-f480064876ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:22.979782 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aea54c-109f-48e3-9c14-f480064876ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:22.980405 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aea54c-109f-48e3-9c14-f480064876ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:22.981956 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aea54c-109f-48e3-9c14-f480064876ba admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:22.993135 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1399/2789] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:22.997004 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a28b1-a5f6-488e-b2f9-ed7cab8e5e7e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:23.006886 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a28b1-a5f6-488e-b2f9-ed7cab8e5e7e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:23.011490 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257bd32a-361e-4e4a-a7be-90a825c16d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:23.012122 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bd32a-361e-4e4a-a7be-90a825c16d0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:23.012401 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bd32a-361e-4e4a-a7be-90a825c16d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:23.012672 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bd32a-361e-4e4a-a7be-90a825c16d0e admin admin] PATH_INFO: `/v3/projects/ba952976205e45628e7357131b8e2c58/users/32719f123903445ca79baa4792db4bd7/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:23.027683 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257bd32a-361e-4e4a-a7be-90a825c16d0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba952976205e45628e7357131b8e2c58, user_id=32719f123903445ca79baa4792db4bd7, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:23.030374 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257bd32a-361e-4e4a-a7be-90a825c16d0e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:23.044296 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1391/2790] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:43:22 2022] PUT /identity/v3/projects/ba952976205e45628e7357131b8e2c58/users/32719f123903445ca79baa4792db4bd7/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:23.106828 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a08d2c-462e-4dd7-8f16-55a2a5016f06 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:23.107138 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a08d2c-462e-4dd7-8f16-55a2a5016f06 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:23.107414 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a08d2c-462e-4dd7-8f16-55a2a5016f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:23.135456 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-58a08d2c-462e-4dd7-8f16-55a2a5016f06 None None] MFA Rules not processed for user `32719f123903445ca79baa4792db4bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:23.136704 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-58a08d2c-462e-4dd7-8f16-55a2a5016f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:23.199092 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1400/2791] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:23.203502 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257bd32a-361e-4e4a-a7be-90a825c16d0e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:23.264273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257bd32a-361e-4e4a-a7be-90a825c16d0e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:23.269433 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04458d3-9a97-4649-8dc8-07c0d61333dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:23.270022 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04458d3-9a97-4649-8dc8-07c0d61333dc admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:23.270331 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04458d3-9a97-4649-8dc8-07c0d61333dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:23.270606 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04458d3-9a97-4649-8dc8-07c0d61333dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:23.271217 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04458d3-9a97-4649-8dc8-07c0d61333dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:23.273112 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04458d3-9a97-4649-8dc8-07c0d61333dc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:23.300130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1392/2792] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:43:23 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:43:23.304182 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a08d2c-462e-4dd7-8f16-55a2a5016f06 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:23.315523 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a08d2c-462e-4dd7-8f16-55a2a5016f06 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:23.373646 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3e643-dad4-413e-9378-66cd59fb0a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:23.374258 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3e643-dad4-413e-9378-66cd59fb0a7a admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:23.374546 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3e643-dad4-413e-9378-66cd59fb0a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:23.374815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3e643-dad4-413e-9378-66cd59fb0a7a admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:23.375469 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3e643-dad4-413e-9378-66cd59fb0a7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:23.377377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3e643-dad4-413e-9378-66cd59fb0a7a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:23.397669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1401/2793] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:43:23 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:43:23.401709 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04458d3-9a97-4649-8dc8-07c0d61333dc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:23.412709 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04458d3-9a97-4649-8dc8-07c0d61333dc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:23.417728 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2147eb-ea7b-46ec-9845-d8f68b85270f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:23.418393 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2147eb-ea7b-46ec-9845-d8f68b85270f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:23.418669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2147eb-ea7b-46ec-9845-d8f68b85270f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:23.418940 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2147eb-ea7b-46ec-9845-d8f68b85270f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:23.419580 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2147eb-ea7b-46ec-9845-d8f68b85270f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:23.421181 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2147eb-ea7b-46ec-9845-d8f68b85270f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:23.432049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1393/2794] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:23.436159 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3e643-dad4-413e-9378-66cd59fb0a7a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:23.446489 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3e643-dad4-413e-9378-66cd59fb0a7a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:23.451139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a016e8-ed79-45a6-bfbe-a80040272257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:23.451787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a016e8-ed79-45a6-bfbe-a80040272257 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:23.452069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a016e8-ed79-45a6-bfbe-a80040272257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:23.452344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a016e8-ed79-45a6-bfbe-a80040272257 admin admin] PATH_INFO: `/v3/projects/5811bf75c9df40b58c2125998383ebf1/users/1cc5a107b54d4e23978d86ee6ebb8b9f/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:23.467457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a016e8-ed79-45a6-bfbe-a80040272257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5811bf75c9df40b58c2125998383ebf1, user_id=1cc5a107b54d4e23978d86ee6ebb8b9f, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:23.470081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a016e8-ed79-45a6-bfbe-a80040272257 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:23.483472 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1402/2795] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:43:23 2022] PUT /identity/v3/projects/5811bf75c9df40b58c2125998383ebf1/users/1cc5a107b54d4e23978d86ee6ebb8b9f/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:23.487542 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2147eb-ea7b-46ec-9845-d8f68b85270f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:23.497541 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2147eb-ea7b-46ec-9845-d8f68b85270f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:23.517015 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:23.554877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6102b-ec39-462b-9f41-bd375897d9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:23.555469 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6102b-ec39-462b-9f41-bd375897d9ae admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:23.555745 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6102b-ec39-462b-9f41-bd375897d9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:23.556013 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6102b-ec39-462b-9f41-bd375897d9ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:23.556642 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd6102b-ec39-462b-9f41-bd375897d9ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:23.558281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd6102b-ec39-462b-9f41-bd375897d9ae admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:23.567966 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1394/2796] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:23.572177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a016e8-ed79-45a6-bfbe-a80040272257 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:23.577520 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:23.584450 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a016e8-ed79-45a6-bfbe-a80040272257 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:23.589531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d9661-4221-453d-b244-1f14ae753a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:23.590289 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d9661-4221-453d-b244-1f14ae753a82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:23.590575 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d9661-4221-453d-b244-1f14ae753a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:23.590852 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d9661-4221-453d-b244-1f14ae753a82 admin admin] PATH_INFO: `/v3/projects/5811bf75c9df40b58c2125998383ebf1/users/1cc5a107b54d4e23978d86ee6ebb8b9f/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:23.592874 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d9661-4221-453d-b244-1f14ae753a82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5811bf75c9df40b58c2125998383ebf1, user_id=1cc5a107b54d4e23978d86ee6ebb8b9f, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:23.595543 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d9661-4221-453d-b244-1f14ae753a82 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:23.606339 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1403/2797] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:43:23 2022] PUT /identity/v3/projects/5811bf75c9df40b58c2125998383ebf1/users/1cc5a107b54d4e23978d86ee6ebb8b9f/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:23.616216 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07978c-e36d-47fa-927c-10b5f6eedad0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:23.616504 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07978c-e36d-47fa-927c-10b5f6eedad0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:23.616781 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07978c-e36d-47fa-927c-10b5f6eedad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:23.645513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2b07978c-e36d-47fa-927c-10b5f6eedad0 None None] MFA Rules not processed for user `1cc5a107b54d4e23978d86ee6ebb8b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:23.646762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2b07978c-e36d-47fa-927c-10b5f6eedad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:23.713097 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1395/2798] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:23.724321 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d9661-4221-453d-b244-1f14ae753a82 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:23.726015 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-610d9661-4221-453d-b244-1f14ae753a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:23.782909 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d9661-4221-453d-b244-1f14ae753a82 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:23.786488 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9c84e-ef7c-4231-bdb2-30283932851e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:23.787099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9c84e-ef7c-4231-bdb2-30283932851e service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:23.787377 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9c84e-ef7c-4231-bdb2-30283932851e service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:23.787649 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9c84e-ef7c-4231-bdb2-30283932851e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:23.796369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9c84e-ef7c-4231-bdb2-30283932851e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:23.797777 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9c84e-ef7c-4231-bdb2-30283932851e service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:23.807370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1404/2799] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:23.881936 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07978c-e36d-47fa-927c-10b5f6eedad0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:23.891662 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07978c-e36d-47fa-927c-10b5f6eedad0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:23.895985 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5c341-3836-41e6-8f02-98603892e091 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:23.896572 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5c341-3836-41e6-8f02-98603892e091 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:23.896847 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5c341-3836-41e6-8f02-98603892e091 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:23.897113 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5c341-3836-41e6-8f02-98603892e091 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:23.951570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5c341-3836-41e6-8f02-98603892e091 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:23.952998 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5c341-3836-41e6-8f02-98603892e091 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:23.962340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1396/2800] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:26.669736 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9c84e-ef7c-4231-bdb2-30283932851e service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:26.731202 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9c84e-ef7c-4231-bdb2-30283932851e service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:26.735739 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e186ac0-cd53-4468-8c69-446230ad863a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:26.736351 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186ac0-cd53-4468-8c69-446230ad863a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:26.736631 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186ac0-cd53-4468-8c69-446230ad863a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:26.736925 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186ac0-cd53-4468-8c69-446230ad863a admin admin] PATH_INFO: `/v3/users/32719f123903445ca79baa4792db4bd7` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:26.738045 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e186ac0-cd53-4468-8c69-446230ad863a admin admin] RBAC: Authorizing `identity:delete_user(user_id=32719f123903445ca79baa4792db4bd7)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:26.739996 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e186ac0-cd53-4468-8c69-446230ad863a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:26.785467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7e186ac0-cd53-4468-8c69-446230ad863a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32719f123903445ca79baa4792db4bd7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:26.792436 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7e186ac0-cd53-4468-8c69-446230ad863a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32719f123903445ca79baa4792db4bd7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:26.799407 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7e186ac0-cd53-4468-8c69-446230ad863a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32719f123903445ca79baa4792db4bd7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:26.800049 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7e186ac0-cd53-4468-8c69-446230ad863a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32719f123903445ca79baa4792db4bd7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:26.800671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7e186ac0-cd53-4468-8c69-446230ad863a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32719f123903445ca79baa4792db4bd7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:26.818744 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1405/2801] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:43:26 2022] DELETE /identity/v3/users/32719f123903445ca79baa4792db4bd7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:26.823008 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10c0c1-ba87-4821-aff1-4197343f5b23 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:26.823508 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10c0c1-ba87-4821-aff1-4197343f5b23 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:26.823614 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10c0c1-ba87-4821-aff1-4197343f5b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:26.842810 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-aa10c0c1-ba87-4821-aff1-4197343f5b23 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:26.844019 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-aa10c0c1-ba87-4821-aff1-4197343f5b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:26.883042 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:26.902454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1397/2802] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:26.912078 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e186ac0-cd53-4468-8c69-446230ad863a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:26.913478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7e186ac0-cd53-4468-8c69-446230ad863a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:26.926834 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:26.972955 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e186ac0-cd53-4468-8c69-446230ad863a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:26.976473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a24ff-e0ba-491c-90f5-e623db6d6448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:26.977062 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a24ff-e0ba-491c-90f5-e623db6d6448 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:26.977335 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a24ff-e0ba-491c-90f5-e623db6d6448 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:26.977608 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a24ff-e0ba-491c-90f5-e623db6d6448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:26.986425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a24ff-e0ba-491c-90f5-e623db6d6448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:26.988871 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a24ff-e0ba-491c-90f5-e623db6d6448 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:26.998094 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1406/2803] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:27.258689 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10c0c1-ba87-4821-aff1-4197343f5b23 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:27.260129 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-aa10c0c1-ba87-4821-aff1-4197343f5b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:27.272314 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10c0c1-ba87-4821-aff1-4197343f5b23 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:27.277298 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dedc27b-0c1f-4241-b461-0fb090be729a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:27.277925 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dedc27b-0c1f-4241-b461-0fb090be729a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:27.278197 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dedc27b-0c1f-4241-b461-0fb090be729a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:27.278509 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dedc27b-0c1f-4241-b461-0fb090be729a admin admin] PATH_INFO: `/v3/projects/ba952976205e45628e7357131b8e2c58` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:27.279612 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dedc27b-0c1f-4241-b461-0fb090be729a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba952976205e45628e7357131b8e2c58)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:27.281535 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dedc27b-0c1f-4241-b461-0fb090be729a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:27.334543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1dedc27b-0c1f-4241-b461-0fb090be729a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba952976205e45628e7357131b8e2c58', 'request_id': 'req-1dedc27b-0c1f-4241-b461-0fb090be729a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:27.342435 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1dedc27b-0c1f-4241-b461-0fb090be729a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba952976205e45628e7357131b8e2c58', 'request_id': 'req-1dedc27b-0c1f-4241-b461-0fb090be729a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:27.352709 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1dedc27b-0c1f-4241-b461-0fb090be729a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba952976205e45628e7357131b8e2c58', 'request_id': 'req-1dedc27b-0c1f-4241-b461-0fb090be729a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:27.367259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1dedc27b-0c1f-4241-b461-0fb090be729a admin admin] The token cache is being invalidate because project ba952976205e45628e7357131b8e2c58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:43:27.367688 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1dedc27b-0c1f-4241-b461-0fb090be729a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:27.368554 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1dedc27b-0c1f-4241-b461-0fb090be729a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:27.370675 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1398/2804] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:43:27 2022] DELETE /identity/v3/projects/ba952976205e45628e7357131b8e2c58 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:27.375725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a24ff-e0ba-491c-90f5-e623db6d6448 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:27.385540 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a24ff-e0ba-491c-90f5-e623db6d6448 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:27.441785 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754aef23-7e35-41ac-bc3f-c4fd35a5d840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:27.442371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754aef23-7e35-41ac-bc3f-c4fd35a5d840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:27.442652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754aef23-7e35-41ac-bc3f-c4fd35a5d840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:27.442924 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754aef23-7e35-41ac-bc3f-c4fd35a5d840 admin admin] PATH_INFO: `/v3/users/1cc5a107b54d4e23978d86ee6ebb8b9f` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:27.444052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754aef23-7e35-41ac-bc3f-c4fd35a5d840 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cc5a107b54d4e23978d86ee6ebb8b9f)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:27.446177 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754aef23-7e35-41ac-bc3f-c4fd35a5d840 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:27.491891 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-754aef23-7e35-41ac-bc3f-c4fd35a5d840 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cc5a107b54d4e23978d86ee6ebb8b9f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:27.498689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-754aef23-7e35-41ac-bc3f-c4fd35a5d840 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cc5a107b54d4e23978d86ee6ebb8b9f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:27.505870 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-754aef23-7e35-41ac-bc3f-c4fd35a5d840 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cc5a107b54d4e23978d86ee6ebb8b9f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:27.506512 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-754aef23-7e35-41ac-bc3f-c4fd35a5d840 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cc5a107b54d4e23978d86ee6ebb8b9f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:27.507122 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-754aef23-7e35-41ac-bc3f-c4fd35a5d840 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cc5a107b54d4e23978d86ee6ebb8b9f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:27.524012 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1407/2805] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:43:27 2022] DELETE /identity/v3/users/1cc5a107b54d4e23978d86ee6ebb8b9f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:27.801113 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dedc27b-0c1f-4241-b461-0fb090be729a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:27.803147 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1dedc27b-0c1f-4241-b461-0fb090be729a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:27.819276 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dedc27b-0c1f-4241-b461-0fb090be729a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:27.881150 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df831816-b38d-4bdb-905a-8d3f84a2ddeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:27.881802 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df831816-b38d-4bdb-905a-8d3f84a2ddeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:27.882039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df831816-b38d-4bdb-905a-8d3f84a2ddeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:27.882350 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df831816-b38d-4bdb-905a-8d3f84a2ddeb admin admin] PATH_INFO: `/v3/projects/5811bf75c9df40b58c2125998383ebf1` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:27.883477 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df831816-b38d-4bdb-905a-8d3f84a2ddeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5811bf75c9df40b58c2125998383ebf1)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:27.885379 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df831816-b38d-4bdb-905a-8d3f84a2ddeb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:27.931635 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-df831816-b38d-4bdb-905a-8d3f84a2ddeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5811bf75c9df40b58c2125998383ebf1', 'request_id': 'req-df831816-b38d-4bdb-905a-8d3f84a2ddeb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:27.937147 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-df831816-b38d-4bdb-905a-8d3f84a2ddeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5811bf75c9df40b58c2125998383ebf1', 'request_id': 'req-df831816-b38d-4bdb-905a-8d3f84a2ddeb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:27.944663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-df831816-b38d-4bdb-905a-8d3f84a2ddeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5811bf75c9df40b58c2125998383ebf1', 'request_id': 'req-df831816-b38d-4bdb-905a-8d3f84a2ddeb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:27.951541 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:27.956864 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-df831816-b38d-4bdb-905a-8d3f84a2ddeb admin admin] The token cache is being invalidate because project 5811bf75c9df40b58c2125998383ebf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:43:27.957340 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-df831816-b38d-4bdb-905a-8d3f84a2ddeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:27.958402 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-df831816-b38d-4bdb-905a-8d3f84a2ddeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:27.960032 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1399/2806] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:43:27 2022] DELETE /identity/v3/projects/5811bf75c9df40b58c2125998383ebf1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:27.964913 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d8d0d-93c1-463b-a4c6-83102ac7feed None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:27.965376 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d8d0d-93c1-463b-a4c6-83102ac7feed None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:27.965487 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d8d0d-93c1-463b-a4c6-83102ac7feed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:27.985513 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-941d8d0d-93c1-463b-a4c6-83102ac7feed None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:27.986694 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-941d8d0d-93c1-463b-a4c6-83102ac7feed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:28.015260 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:28.048824 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1408/2807] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:43:28.057881 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448450d5-ab4d-43fc-88f4-af775103a7d8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:28.058172 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448450d5-ab4d-43fc-88f4-af775103a7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:28.058487 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448450d5-ab4d-43fc-88f4-af775103a7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:28.077979 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-448450d5-ab4d-43fc-88f4-af775103a7d8 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:28.079217 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-448450d5-ab4d-43fc-88f4-af775103a7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:28.087424 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1400/2808] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:28.091482 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d8d0d-93c1-463b-a4c6-83102ac7feed None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:28.101686 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d8d0d-93c1-463b-a4c6-83102ac7feed None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:28.106308 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81adc9-08ee-4c89-ad3a-fbd52db308f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:28.106899 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81adc9-08ee-4c89-ad3a-fbd52db308f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:28.107179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81adc9-08ee-4c89-ad3a-fbd52db308f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:28.107451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81adc9-08ee-4c89-ad3a-fbd52db308f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:28.108099 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81adc9-08ee-4c89-ad3a-fbd52db308f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:43:28.108402 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81adc9-08ee-4c89-ad3a-fbd52db308f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:28.109986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81adc9-08ee-4c89-ad3a-fbd52db308f5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:28.121314 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1409/2809] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:43:28.125468 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448450d5-ab4d-43fc-88f4-af775103a7d8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:28.135072 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448450d5-ab4d-43fc-88f4-af775103a7d8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:28.139676 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbe395-96f2-4b53-9735-c9977bcdb303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:28.140257 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbe395-96f2-4b53-9735-c9977bcdb303 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:28.140536 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbe395-96f2-4b53-9735-c9977bcdb303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:28.140805 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbe395-96f2-4b53-9735-c9977bcdb303 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:28.141430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fbe395-96f2-4b53-9735-c9977bcdb303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:28.144015 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fbe395-96f2-4b53-9735-c9977bcdb303 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:28.170520 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1401/2810] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:43:28 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:43:28.174726 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81adc9-08ee-4c89-ad3a-fbd52db308f5 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:28.184510 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81adc9-08ee-4c89-ad3a-fbd52db308f5 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:28.240390 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c424030-78b5-4d40-a931-646518443bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:28.240919 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c424030-78b5-4d40-a931-646518443bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:28.241199 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c424030-78b5-4d40-a931-646518443bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:28.241469 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c424030-78b5-4d40-a931-646518443bc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:28.242082 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c424030-78b5-4d40-a931-646518443bc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:28.244052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c424030-78b5-4d40-a931-646518443bc2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:28.264644 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1410/2811] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:43:28 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:43:28.268838 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbe395-96f2-4b53-9735-c9977bcdb303 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:28.279508 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbe395-96f2-4b53-9735-c9977bcdb303 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:28.284011 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32506787-36bf-442d-ac50-a8d5d7f727fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:28.284586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32506787-36bf-442d-ac50-a8d5d7f727fe admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:28.284860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32506787-36bf-442d-ac50-a8d5d7f727fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:28.285125 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32506787-36bf-442d-ac50-a8d5d7f727fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:28.285754 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32506787-36bf-442d-ac50-a8d5d7f727fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:28.287359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32506787-36bf-442d-ac50-a8d5d7f727fe admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:28.296918 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1402/2812] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:28.300610 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c424030-78b5-4d40-a931-646518443bc2 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:28.310660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c424030-78b5-4d40-a931-646518443bc2 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:28.316101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c9950-14e2-4f04-bc0d-1c034ac653fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:28.316689 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c9950-14e2-4f04-bc0d-1c034ac653fd admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:28.316966 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c9950-14e2-4f04-bc0d-1c034ac653fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:28.317234 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c9950-14e2-4f04-bc0d-1c034ac653fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:28.317868 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c9950-14e2-4f04-bc0d-1c034ac653fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:28.319511 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c9950-14e2-4f04-bc0d-1c034ac653fd admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:28.330379 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1411/2813] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:28.335412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32506787-36bf-442d-ac50-a8d5d7f727fe admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:28.345746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32506787-36bf-442d-ac50-a8d5d7f727fe admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:28.354540 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695d580-d70d-4397-9b29-60d0321879ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:28.355281 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695d580-d70d-4397-9b29-60d0321879ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:28.355566 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695d580-d70d-4397-9b29-60d0321879ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:28.355822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695d580-d70d-4397-9b29-60d0321879ad admin admin] PATH_INFO: `/v3/projects/1e1a7316dd3140fcbbf9626890b6b6d5/users/91341dbe55ca4fe3bec767012ad56c9a/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:28.371097 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695d580-d70d-4397-9b29-60d0321879ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e1a7316dd3140fcbbf9626890b6b6d5, user_id=91341dbe55ca4fe3bec767012ad56c9a, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:28.373733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695d580-d70d-4397-9b29-60d0321879ad admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:28.388123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1403/2814] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:43:28 2022] PUT /identity/v3/projects/1e1a7316dd3140fcbbf9626890b6b6d5/users/91341dbe55ca4fe3bec767012ad56c9a/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:28.398966 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67de01-efc1-45c1-8ad0-9b5cd607b088 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:28.399256 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67de01-efc1-45c1-8ad0-9b5cd607b088 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:28.399532 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67de01-efc1-45c1-8ad0-9b5cd607b088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:28.427043 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-be67de01-efc1-45c1-8ad0-9b5cd607b088 None None] MFA Rules not processed for user `91341dbe55ca4fe3bec767012ad56c9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:28.428259 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-be67de01-efc1-45c1-8ad0-9b5cd607b088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:28.492117 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1412/2815] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:28.496843 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695d580-d70d-4397-9b29-60d0321879ad admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:28.556721 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695d580-d70d-4397-9b29-60d0321879ad admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:28.561323 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba41f12-65f9-47da-80ff-cba223f2297b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:28.561905 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba41f12-65f9-47da-80ff-cba223f2297b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:28.562179 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba41f12-65f9-47da-80ff-cba223f2297b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:28.562476 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba41f12-65f9-47da-80ff-cba223f2297b admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:28.563078 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba41f12-65f9-47da-80ff-cba223f2297b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:28.565554 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba41f12-65f9-47da-80ff-cba223f2297b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:28.590877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1404/2816] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:43:28 2022] POST /identity/v3/projects => generated 391 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:43:28.594946 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67de01-efc1-45c1-8ad0-9b5cd607b088 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:28.605621 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67de01-efc1-45c1-8ad0-9b5cd607b088 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:28.631986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:28.662872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788692e-e827-4965-90c1-2e27e458d632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:28.663478 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788692e-e827-4965-90c1-2e27e458d632 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:28.663787 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788692e-e827-4965-90c1-2e27e458d632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:28.664060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788692e-e827-4965-90c1-2e27e458d632 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:28.664725 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788692e-e827-4965-90c1-2e27e458d632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:28.666778 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788692e-e827-4965-90c1-2e27e458d632 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:28.689550 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1413/2817] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:43:28 2022] POST /identity/v3/users => generated 302 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:43:28.693996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba41f12-65f9-47da-80ff-cba223f2297b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:28.704629 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba41f12-65f9-47da-80ff-cba223f2297b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:28.711265 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc2bf4-8937-4390-9a39-cf49feebd822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:28.712655 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc2bf4-8937-4390-9a39-cf49feebd822 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:28.712930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc2bf4-8937-4390-9a39-cf49feebd822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:28.713193 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc2bf4-8937-4390-9a39-cf49feebd822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:28.714671 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc2bf4-8937-4390-9a39-cf49feebd822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:28.716256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc2bf4-8937-4390-9a39-cf49feebd822 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:28.726820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1405/2818] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:28.731073 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788692e-e827-4965-90c1-2e27e458d632 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:28.741612 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788692e-e827-4965-90c1-2e27e458d632 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:28.746323 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcccf7a-3086-43fa-b8fc-156196d2f717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:28.746904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcccf7a-3086-43fa-b8fc-156196d2f717 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:28.747183 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcccf7a-3086-43fa-b8fc-156196d2f717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:28.747457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcccf7a-3086-43fa-b8fc-156196d2f717 admin admin] PATH_INFO: `/v3/projects/5da99aca10d14d589763f637e3d8034a/users/2169ad6ba6e748c5b8928df948d09184/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:28.762466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcccf7a-3086-43fa-b8fc-156196d2f717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5da99aca10d14d589763f637e3d8034a, user_id=2169ad6ba6e748c5b8928df948d09184, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:28.765146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcccf7a-3086-43fa-b8fc-156196d2f717 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:28.777547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1414/2819] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:43:28 2022] PUT /identity/v3/projects/5da99aca10d14d589763f637e3d8034a/users/2169ad6ba6e748c5b8928df948d09184/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:28.781401 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc2bf4-8937-4390-9a39-cf49feebd822 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:28.791541 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc2bf4-8937-4390-9a39-cf49feebd822 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:28.848558 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a577ae7-2e41-4830-8a0a-f7f91d397b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:28.849155 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a577ae7-2e41-4830-8a0a-f7f91d397b54 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:28.849434 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a577ae7-2e41-4830-8a0a-f7f91d397b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:28.849704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a577ae7-2e41-4830-8a0a-f7f91d397b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:28.850386 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a577ae7-2e41-4830-8a0a-f7f91d397b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:28.851983 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a577ae7-2e41-4830-8a0a-f7f91d397b54 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:28.861616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1406/2820] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:28.866298 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcccf7a-3086-43fa-b8fc-156196d2f717 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:28.876358 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcccf7a-3086-43fa-b8fc-156196d2f717 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:28.880917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb07d0e-5d6b-43d8-b7da-de4db90bca3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:28.881549 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb07d0e-5d6b-43d8-b7da-de4db90bca3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:28.881829 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb07d0e-5d6b-43d8-b7da-de4db90bca3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:28.882101 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb07d0e-5d6b-43d8-b7da-de4db90bca3c admin admin] PATH_INFO: `/v3/projects/5da99aca10d14d589763f637e3d8034a/users/2169ad6ba6e748c5b8928df948d09184/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:28.884041 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb07d0e-5d6b-43d8-b7da-de4db90bca3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5da99aca10d14d589763f637e3d8034a, user_id=2169ad6ba6e748c5b8928df948d09184, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:28.886692 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb07d0e-5d6b-43d8-b7da-de4db90bca3c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:28.896617 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1415/2821] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:43:28 2022] PUT /identity/v3/projects/5da99aca10d14d589763f637e3d8034a/users/2169ad6ba6e748c5b8928df948d09184/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:28.906486 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbb98c-2e67-43f7-9876-5de177fe891c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:28.906974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbb98c-2e67-43f7-9876-5de177fe891c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:28.907085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbb98c-2e67-43f7-9876-5de177fe891c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:28.915581 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:28.936260 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-efbbb98c-2e67-43f7-9876-5de177fe891c None None] MFA Rules not processed for user `2169ad6ba6e748c5b8928df948d09184`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:28.937485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-efbbb98c-2e67-43f7-9876-5de177fe891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:29.010474 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1407/2822] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:28 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:29.021610 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb07d0e-5d6b-43d8-b7da-de4db90bca3c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:29.023605 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6cb07d0e-5d6b-43d8-b7da-de4db90bca3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:29.082704 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb07d0e-5d6b-43d8-b7da-de4db90bca3c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:29.086344 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec079b-a137-4f3c-b896-39f368b01abc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:29.086933 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec079b-a137-4f3c-b896-39f368b01abc service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:29.087210 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec079b-a137-4f3c-b896-39f368b01abc service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:29.087481 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec079b-a137-4f3c-b896-39f368b01abc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:29.096776 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec079b-a137-4f3c-b896-39f368b01abc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:29.098178 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec079b-a137-4f3c-b896-39f368b01abc service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:29.107429 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1416/2823] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:29 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:29.152595 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbb98c-2e67-43f7-9876-5de177fe891c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:29.153976 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-efbbb98c-2e67-43f7-9876-5de177fe891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:29.166466 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbb98c-2e67-43f7-9876-5de177fe891c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:29.170176 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c421608-b61c-4537-966e-2106b91cc5b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:29.170801 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421608-b61c-4537-966e-2106b91cc5b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:29.171077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421608-b61c-4537-966e-2106b91cc5b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:29.171342 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421608-b61c-4537-966e-2106b91cc5b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:29.226447 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c421608-b61c-4537-966e-2106b91cc5b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:29.227869 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c421608-b61c-4537-966e-2106b91cc5b5 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:29.236716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1408/2824] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:29 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:43:32.486820 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec079b-a137-4f3c-b896-39f368b01abc service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:32.548370 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec079b-a137-4f3c-b896-39f368b01abc service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:32.552878 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c87f2-2d5c-4ecc-91a8-f4e20ccd8002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:32.553486 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c87f2-2d5c-4ecc-91a8-f4e20ccd8002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:32.553760 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c87f2-2d5c-4ecc-91a8-f4e20ccd8002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:32.554030 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c87f2-2d5c-4ecc-91a8-f4e20ccd8002 admin admin] PATH_INFO: `/v3/users/91341dbe55ca4fe3bec767012ad56c9a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:32.555147 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c87f2-2d5c-4ecc-91a8-f4e20ccd8002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91341dbe55ca4fe3bec767012ad56c9a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:32.557022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c87f2-2d5c-4ecc-91a8-f4e20ccd8002 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:32.583767 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:32.603938 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5d5c87f2-2d5c-4ecc-91a8-f4e20ccd8002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91341dbe55ca4fe3bec767012ad56c9a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:32.610659 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5d5c87f2-2d5c-4ecc-91a8-f4e20ccd8002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91341dbe55ca4fe3bec767012ad56c9a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:32.617850 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5d5c87f2-2d5c-4ecc-91a8-f4e20ccd8002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91341dbe55ca4fe3bec767012ad56c9a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:32.618491 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5d5c87f2-2d5c-4ecc-91a8-f4e20ccd8002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91341dbe55ca4fe3bec767012ad56c9a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:32.619090 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5d5c87f2-2d5c-4ecc-91a8-f4e20ccd8002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91341dbe55ca4fe3bec767012ad56c9a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:32.637367 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1417/2825] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:43:32 2022] DELETE /identity/v3/users/91341dbe55ca4fe3bec767012ad56c9a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:32.641818 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922796f-9554-47ac-a1e7-32d1a8ff327d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:32.642345 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922796f-9554-47ac-a1e7-32d1a8ff327d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:32.642461 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922796f-9554-47ac-a1e7-32d1a8ff327d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:32.663303 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7922796f-9554-47ac-a1e7-32d1a8ff327d None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:32.664614 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7922796f-9554-47ac-a1e7-32d1a8ff327d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:32.722453 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1409/2826] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:32.732543 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c87f2-2d5c-4ecc-91a8-f4e20ccd8002 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:32.734019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5d5c87f2-2d5c-4ecc-91a8-f4e20ccd8002 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:32.791170 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c87f2-2d5c-4ecc-91a8-f4e20ccd8002 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:32.794905 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999c9e9-f975-45e2-b8f0-24f4465ce23a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:32.795501 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999c9e9-f975-45e2-b8f0-24f4465ce23a service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:32.795807 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999c9e9-f975-45e2-b8f0-24f4465ce23a service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:32.796073 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999c9e9-f975-45e2-b8f0-24f4465ce23a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:32.804870 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999c9e9-f975-45e2-b8f0-24f4465ce23a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:32.806306 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999c9e9-f975-45e2-b8f0-24f4465ce23a service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:32.816372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1418/2827] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:33.097280 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922796f-9554-47ac-a1e7-32d1a8ff327d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:33.098960 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7922796f-9554-47ac-a1e7-32d1a8ff327d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:33.113472 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922796f-9554-47ac-a1e7-32d1a8ff327d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:33.118442 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6278f3-f90c-48bd-859c-a6f804c184f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:33.120119 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6278f3-f90c-48bd-859c-a6f804c184f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:33.120541 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6278f3-f90c-48bd-859c-a6f804c184f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:33.120716 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6278f3-f90c-48bd-859c-a6f804c184f5 admin admin] PATH_INFO: `/v3/projects/1e1a7316dd3140fcbbf9626890b6b6d5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:33.121841 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6278f3-f90c-48bd-859c-a6f804c184f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e1a7316dd3140fcbbf9626890b6b6d5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:33.123784 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6278f3-f90c-48bd-859c-a6f804c184f5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:33.149305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:33.170824 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5d6278f3-f90c-48bd-859c-a6f804c184f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e1a7316dd3140fcbbf9626890b6b6d5', 'request_id': 'req-5d6278f3-f90c-48bd-859c-a6f804c184f5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:33.176567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5d6278f3-f90c-48bd-859c-a6f804c184f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e1a7316dd3140fcbbf9626890b6b6d5', 'request_id': 'req-5d6278f3-f90c-48bd-859c-a6f804c184f5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:33.183662 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5d6278f3-f90c-48bd-859c-a6f804c184f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e1a7316dd3140fcbbf9626890b6b6d5', 'request_id': 'req-5d6278f3-f90c-48bd-859c-a6f804c184f5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:33.195496 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5d6278f3-f90c-48bd-859c-a6f804c184f5 admin admin] The token cache is being invalidate because project 1e1a7316dd3140fcbbf9626890b6b6d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:43:33.195785 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5d6278f3-f90c-48bd-859c-a6f804c184f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:33.196415 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5d6278f3-f90c-48bd-859c-a6f804c184f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:33.197883 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1410/2828] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:43:33 2022] DELETE /identity/v3/projects/1e1a7316dd3140fcbbf9626890b6b6d5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:33.201874 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999c9e9-f975-45e2-b8f0-24f4465ce23a service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:33.211534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999c9e9-f975-45e2-b8f0-24f4465ce23a service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:33.267817 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955df5a-cf1e-4ecd-b8fd-a4d0db32e13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:33.268440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955df5a-cf1e-4ecd-b8fd-a4d0db32e13a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:33.268717 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955df5a-cf1e-4ecd-b8fd-a4d0db32e13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:33.268988 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955df5a-cf1e-4ecd-b8fd-a4d0db32e13a admin admin] PATH_INFO: `/v3/users/2169ad6ba6e748c5b8928df948d09184` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:33.270094 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955df5a-cf1e-4ecd-b8fd-a4d0db32e13a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2169ad6ba6e748c5b8928df948d09184)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:33.272137 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955df5a-cf1e-4ecd-b8fd-a4d0db32e13a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:33.317571 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f955df5a-cf1e-4ecd-b8fd-a4d0db32e13a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2169ad6ba6e748c5b8928df948d09184'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:33.318780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:33.326692 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f955df5a-cf1e-4ecd-b8fd-a4d0db32e13a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2169ad6ba6e748c5b8928df948d09184'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:33.334797 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f955df5a-cf1e-4ecd-b8fd-a4d0db32e13a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2169ad6ba6e748c5b8928df948d09184'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:33.335399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f955df5a-cf1e-4ecd-b8fd-a4d0db32e13a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2169ad6ba6e748c5b8928df948d09184'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:33.336032 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f955df5a-cf1e-4ecd-b8fd-a4d0db32e13a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2169ad6ba6e748c5b8928df948d09184'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:33.353138 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1419/2829] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:43:33 2022] DELETE /identity/v3/users/2169ad6ba6e748c5b8928df948d09184 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:33.618893 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6278f3-f90c-48bd-859c-a6f804c184f5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:33.620305 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5d6278f3-f90c-48bd-859c-a6f804c184f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:33.634546 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6278f3-f90c-48bd-859c-a6f804c184f5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:33.689644 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b227f6-27c6-4f59-bd8f-19854621db1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:33.698092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b227f6-27c6-4f59-bd8f-19854621db1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:33.698092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b227f6-27c6-4f59-bd8f-19854621db1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:33.698092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b227f6-27c6-4f59-bd8f-19854621db1d admin admin] PATH_INFO: `/v3/projects/5da99aca10d14d589763f637e3d8034a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:33.698092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b227f6-27c6-4f59-bd8f-19854621db1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5da99aca10d14d589763f637e3d8034a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:33.698092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b227f6-27c6-4f59-bd8f-19854621db1d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:33.741836 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d8b227f6-27c6-4f59-bd8f-19854621db1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5da99aca10d14d589763f637e3d8034a', 'request_id': 'req-d8b227f6-27c6-4f59-bd8f-19854621db1d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:33.747427 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d8b227f6-27c6-4f59-bd8f-19854621db1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5da99aca10d14d589763f637e3d8034a', 'request_id': 'req-d8b227f6-27c6-4f59-bd8f-19854621db1d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:33.754333 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d8b227f6-27c6-4f59-bd8f-19854621db1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5da99aca10d14d589763f637e3d8034a', 'request_id': 'req-d8b227f6-27c6-4f59-bd8f-19854621db1d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:33.765156 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d8b227f6-27c6-4f59-bd8f-19854621db1d admin admin] The token cache is being invalidate because project 5da99aca10d14d589763f637e3d8034a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:43:33.765439 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d8b227f6-27c6-4f59-bd8f-19854621db1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:33.766054 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d8b227f6-27c6-4f59-bd8f-19854621db1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:33.768795 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1411/2830] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:43:33 2022] DELETE /identity/v3/projects/5da99aca10d14d589763f637e3d8034a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:33.773730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63131286-178a-4a42-bc0d-6b61ef692de2 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:33.774226 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63131286-178a-4a42-bc0d-6b61ef692de2 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:33.774363 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63131286-178a-4a42-bc0d-6b61ef692de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:33.793996 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-63131286-178a-4a42-bc0d-6b61ef692de2 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:33.795238 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-63131286-178a-4a42-bc0d-6b61ef692de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:33.853578 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1420/2831] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:43:33.861086 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801a9c1-ee04-4ac5-aec0-1dd906c3027e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:33.861363 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801a9c1-ee04-4ac5-aec0-1dd906c3027e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:33.861631 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801a9c1-ee04-4ac5-aec0-1dd906c3027e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:33.880528 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0801a9c1-ee04-4ac5-aec0-1dd906c3027e None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:33.881711 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0801a9c1-ee04-4ac5-aec0-1dd906c3027e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:33.889414 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1412/2832] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:33.894044 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63131286-178a-4a42-bc0d-6b61ef692de2 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:33.903671 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63131286-178a-4a42-bc0d-6b61ef692de2 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:33.908206 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545f4af-651d-4d5e-aada-07abb3712760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:33.909399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545f4af-651d-4d5e-aada-07abb3712760 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:33.909677 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545f4af-651d-4d5e-aada-07abb3712760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:33.909946 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545f4af-651d-4d5e-aada-07abb3712760 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:33.910652 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545f4af-651d-4d5e-aada-07abb3712760 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:43:33.910954 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545f4af-651d-4d5e-aada-07abb3712760 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:33.912568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545f4af-651d-4d5e-aada-07abb3712760 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:33.924329 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1421/2833] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:43:33.928571 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801a9c1-ee04-4ac5-aec0-1dd906c3027e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:33.937636 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801a9c1-ee04-4ac5-aec0-1dd906c3027e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:33.942524 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd45bcc-1b96-4ccd-9c95-ab7d869949e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:33.943099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd45bcc-1b96-4ccd-9c95-ab7d869949e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:33.943376 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd45bcc-1b96-4ccd-9c95-ab7d869949e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:33.943642 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd45bcc-1b96-4ccd-9c95-ab7d869949e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:33.944251 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd45bcc-1b96-4ccd-9c95-ab7d869949e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:33.946128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd45bcc-1b96-4ccd-9c95-ab7d869949e7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:33.973524 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1413/2834] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:43:33 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:43:33.977206 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545f4af-651d-4d5e-aada-07abb3712760 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:33.987304 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545f4af-651d-4d5e-aada-07abb3712760 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:34.041935 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb6a5d-1943-4952-8cae-d102c01f6a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:34.042556 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb6a5d-1943-4952-8cae-d102c01f6a99 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:34.042830 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb6a5d-1943-4952-8cae-d102c01f6a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:34.043097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb6a5d-1943-4952-8cae-d102c01f6a99 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:34.043708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb6a5d-1943-4952-8cae-d102c01f6a99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:34.045613 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb6a5d-1943-4952-8cae-d102c01f6a99 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:34.065996 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1422/2835] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:43:33 2022] POST /identity/v3/users => generated 295 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:43:34.070113 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd45bcc-1b96-4ccd-9c95-ab7d869949e7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:34.080319 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd45bcc-1b96-4ccd-9c95-ab7d869949e7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:34.085397 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1807fdb-142b-4263-8bed-665592f3df53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:34.085975 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1807fdb-142b-4263-8bed-665592f3df53 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:34.086268 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1807fdb-142b-4263-8bed-665592f3df53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:34.086552 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1807fdb-142b-4263-8bed-665592f3df53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:34.087185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1807fdb-142b-4263-8bed-665592f3df53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:34.088764 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1807fdb-142b-4263-8bed-665592f3df53 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:34.099047 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1414/2836] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:34.102682 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb6a5d-1943-4952-8cae-d102c01f6a99 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:34.112534 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb6a5d-1943-4952-8cae-d102c01f6a99 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:34.117064 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a2ebe-21b4-4a42-90c6-f0f05b47173c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:34.117634 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a2ebe-21b4-4a42-90c6-f0f05b47173c admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:34.117909 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a2ebe-21b4-4a42-90c6-f0f05b47173c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:34.118179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a2ebe-21b4-4a42-90c6-f0f05b47173c admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:34.118836 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a2ebe-21b4-4a42-90c6-f0f05b47173c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:34.120425 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a2ebe-21b4-4a42-90c6-f0f05b47173c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:34.131220 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1423/2837] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:34.135197 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1807fdb-142b-4263-8bed-665592f3df53 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:34.140513 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:34.147364 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1807fdb-142b-4263-8bed-665592f3df53 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:34.153058 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f56551-9b91-4f2f-a25a-84d2c91a5e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:34.153710 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f56551-9b91-4f2f-a25a-84d2c91a5e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:34.153994 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f56551-9b91-4f2f-a25a-84d2c91a5e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:34.154288 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f56551-9b91-4f2f-a25a-84d2c91a5e69 admin admin] PATH_INFO: `/v3/projects/0f3a3536adac4beeb163c0e50b37b018/users/ee02d4aaf852456cb24f2d7b9369cad9/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:34.168728 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f56551-9b91-4f2f-a25a-84d2c91a5e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f3a3536adac4beeb163c0e50b37b018, user_id=ee02d4aaf852456cb24f2d7b9369cad9, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:34.171426 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f56551-9b91-4f2f-a25a-84d2c91a5e69 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:34.183386 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1415/2838] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:43:34 2022] PUT /identity/v3/projects/0f3a3536adac4beeb163c0e50b37b018/users/ee02d4aaf852456cb24f2d7b9369cad9/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:34.194393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee0cb8-aea0-4f92-b6fa-4500823c03db None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:34.194643 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee0cb8-aea0-4f92-b6fa-4500823c03db None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:34.194913 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee0cb8-aea0-4f92-b6fa-4500823c03db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:34.222550 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-06ee0cb8-aea0-4f92-b6fa-4500823c03db None None] MFA Rules not processed for user `ee02d4aaf852456cb24f2d7b9369cad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:34.223802 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-06ee0cb8-aea0-4f92-b6fa-4500823c03db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:34.242083 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:34.290043 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1424/2839] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:34.294651 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f56551-9b91-4f2f-a25a-84d2c91a5e69 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:34.355303 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f56551-9b91-4f2f-a25a-84d2c91a5e69 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:34.359854 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5108d5c-3468-4e81-965c-1a7d5923d517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:34.360438 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5108d5c-3468-4e81-965c-1a7d5923d517 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:34.360714 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5108d5c-3468-4e81-965c-1a7d5923d517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:34.360982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5108d5c-3468-4e81-965c-1a7d5923d517 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:34.361588 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5108d5c-3468-4e81-965c-1a7d5923d517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:34.363508 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5108d5c-3468-4e81-965c-1a7d5923d517 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:34.389137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1416/2840] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:43:34 2022] POST /identity/v3/projects => generated 389 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:43:34.393835 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee0cb8-aea0-4f92-b6fa-4500823c03db None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:34.404371 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee0cb8-aea0-4f92-b6fa-4500823c03db None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:34.461261 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150dea97-41fe-4f24-9ce1-2decd5b627c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:34.461862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150dea97-41fe-4f24-9ce1-2decd5b627c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:34.462136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150dea97-41fe-4f24-9ce1-2decd5b627c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:34.462452 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150dea97-41fe-4f24-9ce1-2decd5b627c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:34.463084 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150dea97-41fe-4f24-9ce1-2decd5b627c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:34.464962 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150dea97-41fe-4f24-9ce1-2decd5b627c3 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:34.485372 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1425/2841] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:43:34 2022] POST /identity/v3/users => generated 301 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:43:34.489783 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5108d5c-3468-4e81-965c-1a7d5923d517 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:34.500077 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5108d5c-3468-4e81-965c-1a7d5923d517 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:34.505271 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b7603-acd1-4be4-96a5-09fa4ddaa028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:34.505860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b7603-acd1-4be4-96a5-09fa4ddaa028 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:34.506140 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b7603-acd1-4be4-96a5-09fa4ddaa028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:34.506446 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b7603-acd1-4be4-96a5-09fa4ddaa028 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:34.507087 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b7603-acd1-4be4-96a5-09fa4ddaa028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:34.508675 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b7603-acd1-4be4-96a5-09fa4ddaa028 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:34.518961 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1417/2842] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:34.522965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150dea97-41fe-4f24-9ce1-2decd5b627c3 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:34.532838 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150dea97-41fe-4f24-9ce1-2decd5b627c3 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:34.537347 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6db69-4c0f-40b2-8e90-91f1c2a0d0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:34.538019 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6db69-4c0f-40b2-8e90-91f1c2a0d0ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:34.538306 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6db69-4c0f-40b2-8e90-91f1c2a0d0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:34.538579 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6db69-4c0f-40b2-8e90-91f1c2a0d0ab admin admin] PATH_INFO: `/v3/projects/43567747b15e4e48b82031006bb51776/users/000c73356536408b9098b6200caef892/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:34.553368 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6db69-4c0f-40b2-8e90-91f1c2a0d0ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=43567747b15e4e48b82031006bb51776, user_id=000c73356536408b9098b6200caef892, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:34.556115 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6db69-4c0f-40b2-8e90-91f1c2a0d0ab admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:34.569708 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1426/2843] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:43:34 2022] PUT /identity/v3/projects/43567747b15e4e48b82031006bb51776/users/000c73356536408b9098b6200caef892/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:34.573702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b7603-acd1-4be4-96a5-09fa4ddaa028 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:34.583854 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b7603-acd1-4be4-96a5-09fa4ddaa028 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:34.637903 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02546c6-1c0f-426b-b1ec-10926c9784bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:34.638525 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02546c6-1c0f-426b-b1ec-10926c9784bd admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:34.638808 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02546c6-1c0f-426b-b1ec-10926c9784bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:34.639076 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02546c6-1c0f-426b-b1ec-10926c9784bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:34.639805 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02546c6-1c0f-426b-b1ec-10926c9784bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:34.641391 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02546c6-1c0f-426b-b1ec-10926c9784bd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:34.651084 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1418/2844] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:34.655198 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6db69-4c0f-40b2-8e90-91f1c2a0d0ab admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:34.665240 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6db69-4c0f-40b2-8e90-91f1c2a0d0ab admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:34.669753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778f387-93f8-4851-bb07-f85243279c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:34.670424 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778f387-93f8-4851-bb07-f85243279c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:34.670701 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778f387-93f8-4851-bb07-f85243279c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:34.670968 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778f387-93f8-4851-bb07-f85243279c2c admin admin] PATH_INFO: `/v3/projects/43567747b15e4e48b82031006bb51776/users/000c73356536408b9098b6200caef892/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:34.672853 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778f387-93f8-4851-bb07-f85243279c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=43567747b15e4e48b82031006bb51776, user_id=000c73356536408b9098b6200caef892, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:34.675975 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778f387-93f8-4851-bb07-f85243279c2c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:34.685818 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1427/2845] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:43:34 2022] PUT /identity/v3/projects/43567747b15e4e48b82031006bb51776/users/000c73356536408b9098b6200caef892/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:43:34.697641 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345b98c-59d4-4204-89f2-89df2884776b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:34.698128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345b98c-59d4-4204-89f2-89df2884776b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:34.698264 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345b98c-59d4-4204-89f2-89df2884776b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:34.725611 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9345b98c-59d4-4204-89f2-89df2884776b None None] MFA Rules not processed for user `000c73356536408b9098b6200caef892`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:34.726859 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9345b98c-59d4-4204-89f2-89df2884776b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:34.765205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:34.795188 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1419/2846] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:34 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:34.806314 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778f387-93f8-4851-bb07-f85243279c2c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:34.808094 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4778f387-93f8-4851-bb07-f85243279c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:34.869926 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778f387-93f8-4851-bb07-f85243279c2c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:34.873548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda8f18-1dd7-432e-9fe1-32ccd52163fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:34.874146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda8f18-1dd7-432e-9fe1-32ccd52163fc service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:34.874460 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda8f18-1dd7-432e-9fe1-32ccd52163fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:34.874730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda8f18-1dd7-432e-9fe1-32ccd52163fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:34.883428 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda8f18-1dd7-432e-9fe1-32ccd52163fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:34.884849 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda8f18-1dd7-432e-9fe1-32ccd52163fc service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:34.894135 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1428/2847] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:34 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:35.027099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345b98c-59d4-4204-89f2-89df2884776b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:35.087480 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345b98c-59d4-4204-89f2-89df2884776b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:35.091877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafb480-44ae-41d6-80cd-0a17dd164713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:35.092479 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafb480-44ae-41d6-80cd-0a17dd164713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:35.092750 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafb480-44ae-41d6-80cd-0a17dd164713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:35.093012 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafb480-44ae-41d6-80cd-0a17dd164713 admin admin] PATH_INFO: `/v3/users/ee02d4aaf852456cb24f2d7b9369cad9` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:35.094101 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baafb480-44ae-41d6-80cd-0a17dd164713 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee02d4aaf852456cb24f2d7b9369cad9)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:35.096523 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baafb480-44ae-41d6-80cd-0a17dd164713 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:35.141038 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-baafb480-44ae-41d6-80cd-0a17dd164713 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee02d4aaf852456cb24f2d7b9369cad9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:35.147974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-baafb480-44ae-41d6-80cd-0a17dd164713 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee02d4aaf852456cb24f2d7b9369cad9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:35.154683 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-baafb480-44ae-41d6-80cd-0a17dd164713 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee02d4aaf852456cb24f2d7b9369cad9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:35.155297 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-baafb480-44ae-41d6-80cd-0a17dd164713 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee02d4aaf852456cb24f2d7b9369cad9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:35.155899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-baafb480-44ae-41d6-80cd-0a17dd164713 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee02d4aaf852456cb24f2d7b9369cad9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:35.172973 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1420/2848] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:43:35 2022] DELETE /identity/v3/users/ee02d4aaf852456cb24f2d7b9369cad9 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:35.178038 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee469e11-2ae2-441e-b672-82a30704be7c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:35.178422 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee469e11-2ae2-441e-b672-82a30704be7c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:35.178726 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee469e11-2ae2-441e-b672-82a30704be7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:35.197169 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ee469e11-2ae2-441e-b672-82a30704be7c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:35.198467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ee469e11-2ae2-441e-b672-82a30704be7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:35.207466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:35.258473 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1429/2849] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:35.268791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafb480-44ae-41d6-80cd-0a17dd164713 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:35.270174 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-baafb480-44ae-41d6-80cd-0a17dd164713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:35.325245 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafb480-44ae-41d6-80cd-0a17dd164713 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:35.328771 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f66de-88c0-4d56-b526-6dc8bad84378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:35.329359 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f66de-88c0-4d56-b526-6dc8bad84378 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:35.329631 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f66de-88c0-4d56-b526-6dc8bad84378 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:35.329896 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f66de-88c0-4d56-b526-6dc8bad84378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:35.337123 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:35.339450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f66de-88c0-4d56-b526-6dc8bad84378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:35.341259 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f66de-88c0-4d56-b526-6dc8bad84378 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:35.351994 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1421/2850] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:35.616970 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee469e11-2ae2-441e-b672-82a30704be7c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:35.618447 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ee469e11-2ae2-441e-b672-82a30704be7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:35.632153 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee469e11-2ae2-441e-b672-82a30704be7c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:35.638133 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad7a8c-94d4-4965-acb5-342ebc2c00e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:35.638825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad7a8c-94d4-4965-acb5-342ebc2c00e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:35.639130 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad7a8c-94d4-4965-acb5-342ebc2c00e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:35.639402 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad7a8c-94d4-4965-acb5-342ebc2c00e6 admin admin] PATH_INFO: `/v3/projects/0f3a3536adac4beeb163c0e50b37b018` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:35.640521 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad7a8c-94d4-4965-acb5-342ebc2c00e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f3a3536adac4beeb163c0e50b37b018)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:35.643888 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad7a8c-94d4-4965-acb5-342ebc2c00e6 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:35.688326 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-6bad7a8c-94d4-4965-acb5-342ebc2c00e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f3a3536adac4beeb163c0e50b37b018', 'request_id': 'req-6bad7a8c-94d4-4965-acb5-342ebc2c00e6'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:35.694430 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-6bad7a8c-94d4-4965-acb5-342ebc2c00e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f3a3536adac4beeb163c0e50b37b018', 'request_id': 'req-6bad7a8c-94d4-4965-acb5-342ebc2c00e6'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:35.701409 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-6bad7a8c-94d4-4965-acb5-342ebc2c00e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f3a3536adac4beeb163c0e50b37b018', 'request_id': 'req-6bad7a8c-94d4-4965-acb5-342ebc2c00e6'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:35.713112 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-6bad7a8c-94d4-4965-acb5-342ebc2c00e6 admin admin] The token cache is being invalidate because project 0f3a3536adac4beeb163c0e50b37b018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:43:35.713401 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-6bad7a8c-94d4-4965-acb5-342ebc2c00e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:35.714032 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-6bad7a8c-94d4-4965-acb5-342ebc2c00e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:35.715590 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1430/2851] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:43:35 2022] DELETE /identity/v3/projects/0f3a3536adac4beeb163c0e50b37b018 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:35.719850 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f66de-88c0-4d56-b526-6dc8bad84378 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:35.729472 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f66de-88c0-4d56-b526-6dc8bad84378 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:35.784309 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b61d7-7a32-471f-b808-c8284407f9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:35.784929 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b61d7-7a32-471f-b808-c8284407f9c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:35.785209 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b61d7-7a32-471f-b808-c8284407f9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:35.785478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b61d7-7a32-471f-b808-c8284407f9c2 admin admin] PATH_INFO: `/v3/users/000c73356536408b9098b6200caef892` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:35.786617 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b61d7-7a32-471f-b808-c8284407f9c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=000c73356536408b9098b6200caef892)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:35.788534 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b61d7-7a32-471f-b808-c8284407f9c2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:35.833530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-352b61d7-7a32-471f-b808-c8284407f9c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '000c73356536408b9098b6200caef892'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:35.840914 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-352b61d7-7a32-471f-b808-c8284407f9c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '000c73356536408b9098b6200caef892'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:35.847886 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-352b61d7-7a32-471f-b808-c8284407f9c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '000c73356536408b9098b6200caef892'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:35.848494 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-352b61d7-7a32-471f-b808-c8284407f9c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '000c73356536408b9098b6200caef892'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:35.849083 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-352b61d7-7a32-471f-b808-c8284407f9c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '000c73356536408b9098b6200caef892'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:35.865721 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1422/2852] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:43:35 2022] DELETE /identity/v3/users/000c73356536408b9098b6200caef892 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:36.129870 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad7a8c-94d4-4965-acb5-342ebc2c00e6 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:36.131034 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6bad7a8c-94d4-4965-acb5-342ebc2c00e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:36.151718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad7a8c-94d4-4965-acb5-342ebc2c00e6 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:36.207714 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f4dbc-fc0d-41e4-80d3-ae1ab049e40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:36.208329 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f4dbc-fc0d-41e4-80d3-ae1ab049e40b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:36.208610 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f4dbc-fc0d-41e4-80d3-ae1ab049e40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:36.208875 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f4dbc-fc0d-41e4-80d3-ae1ab049e40b admin admin] PATH_INFO: `/v3/projects/43567747b15e4e48b82031006bb51776` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:36.209965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f4dbc-fc0d-41e4-80d3-ae1ab049e40b admin admin] RBAC: Authorizing `identity:delete_project(project_id=43567747b15e4e48b82031006bb51776)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:36.211901 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f4dbc-fc0d-41e4-80d3-ae1ab049e40b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:36.254188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:36.261750 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f27f4dbc-fc0d-41e4-80d3-ae1ab049e40b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43567747b15e4e48b82031006bb51776', 'request_id': 'req-f27f4dbc-fc0d-41e4-80d3-ae1ab049e40b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:36.268664 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f27f4dbc-fc0d-41e4-80d3-ae1ab049e40b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43567747b15e4e48b82031006bb51776', 'request_id': 'req-f27f4dbc-fc0d-41e4-80d3-ae1ab049e40b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:36.275924 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f27f4dbc-fc0d-41e4-80d3-ae1ab049e40b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43567747b15e4e48b82031006bb51776', 'request_id': 'req-f27f4dbc-fc0d-41e4-80d3-ae1ab049e40b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:36.287124 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f27f4dbc-fc0d-41e4-80d3-ae1ab049e40b admin admin] The token cache is being invalidate because project 43567747b15e4e48b82031006bb51776 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:43:36.287417 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f27f4dbc-fc0d-41e4-80d3-ae1ab049e40b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:36.288060 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f27f4dbc-fc0d-41e4-80d3-ae1ab049e40b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:36.289568 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1431/2853] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:43:36 2022] DELETE /identity/v3/projects/43567747b15e4e48b82031006bb51776 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:36.294404 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe64e3-4d69-4524-ae7f-5d775bee54fb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:36.294693 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe64e3-4d69-4524-ae7f-5d775bee54fb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:36.294963 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe64e3-4d69-4524-ae7f-5d775bee54fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:36.314273 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-91fe64e3-4d69-4524-ae7f-5d775bee54fb None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:36.316360 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-91fe64e3-4d69-4524-ae7f-5d775bee54fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:36.373199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1423/2854] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:36.381183 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049a42d-9bd6-49ea-931c-f937694c85f1 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:36.381463 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049a42d-9bd6-49ea-931c-f937694c85f1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:36.381730 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049a42d-9bd6-49ea-931c-f937694c85f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:36.401138 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-e049a42d-9bd6-49ea-931c-f937694c85f1 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:36.402508 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e049a42d-9bd6-49ea-931c-f937694c85f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:36.411856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1432/2855] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:36.418539 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe64e3-4d69-4524-ae7f-5d775bee54fb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:36.429220 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe64e3-4d69-4524-ae7f-5d775bee54fb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:36.433980 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fb7ca-8706-45fc-9ebf-b3c4b03ad401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:36.434591 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fb7ca-8706-45fc-9ebf-b3c4b03ad401 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:36.434867 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fb7ca-8706-45fc-9ebf-b3c4b03ad401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:36.435131 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fb7ca-8706-45fc-9ebf-b3c4b03ad401 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:36.435766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fb7ca-8706-45fc-9ebf-b3c4b03ad401 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:43:36.436064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fb7ca-8706-45fc-9ebf-b3c4b03ad401 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:36.437639 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fb7ca-8706-45fc-9ebf-b3c4b03ad401 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:36.450111 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1424/2856] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:43:36.454476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049a42d-9bd6-49ea-931c-f937694c85f1 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:36.463694 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049a42d-9bd6-49ea-931c-f937694c85f1 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:36.468321 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc773a0a-1368-4ee5-bbfb-9463502cd112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:36.468903 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc773a0a-1368-4ee5-bbfb-9463502cd112 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:36.469183 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc773a0a-1368-4ee5-bbfb-9463502cd112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:36.469456 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc773a0a-1368-4ee5-bbfb-9463502cd112 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:36.470079 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc773a0a-1368-4ee5-bbfb-9463502cd112 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:36.472486 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc773a0a-1368-4ee5-bbfb-9463502cd112 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:36.499077 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1433/2857] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:43:36 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:43:36.503189 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fb7ca-8706-45fc-9ebf-b3c4b03ad401 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:36.508302 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:36.514389 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fb7ca-8706-45fc-9ebf-b3c4b03ad401 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:36.571205 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdec3a9-1837-4c87-994c-840130ff448e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:36.571823 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdec3a9-1837-4c87-994c-840130ff448e admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:36.572096 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdec3a9-1837-4c87-994c-840130ff448e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:36.572364 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdec3a9-1837-4c87-994c-840130ff448e admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:36.572970 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdec3a9-1837-4c87-994c-840130ff448e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:36.574921 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdec3a9-1837-4c87-994c-840130ff448e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:36.594865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1425/2858] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:43:36 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:43:36.599514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc773a0a-1368-4ee5-bbfb-9463502cd112 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:36.609709 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc773a0a-1368-4ee5-bbfb-9463502cd112 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:36.614987 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ae701-6a16-4235-b2b2-6864e82467ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:36.615570 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ae701-6a16-4235-b2b2-6864e82467ba admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:36.615847 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ae701-6a16-4235-b2b2-6864e82467ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:36.616120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ae701-6a16-4235-b2b2-6864e82467ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:36.616758 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ae701-6a16-4235-b2b2-6864e82467ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:36.618364 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ae701-6a16-4235-b2b2-6864e82467ba admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:36.628061 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1434/2859] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:36.631822 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdec3a9-1837-4c87-994c-840130ff448e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:36.641887 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdec3a9-1837-4c87-994c-840130ff448e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:36.646530 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dec132-9c53-4622-a851-1099fd581221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:36.647110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dec132-9c53-4622-a851-1099fd581221 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:36.647385 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dec132-9c53-4622-a851-1099fd581221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:36.647651 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dec132-9c53-4622-a851-1099fd581221 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:36.648274 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dec132-9c53-4622-a851-1099fd581221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:36.649848 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dec132-9c53-4622-a851-1099fd581221 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:36.660617 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1426/2860] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:36.664592 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ae701-6a16-4235-b2b2-6864e82467ba admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:36.674888 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ae701-6a16-4235-b2b2-6864e82467ba admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:36.679485 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb737b-572d-4f90-b160-98dfa2967fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:36.680111 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb737b-572d-4f90-b160-98dfa2967fe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:36.680388 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb737b-572d-4f90-b160-98dfa2967fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:36.680658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb737b-572d-4f90-b160-98dfa2967fe0 admin admin] PATH_INFO: `/v3/projects/8d13a5d809e74cea8053c918f6ece4d3/users/5e2504d045d247f584d294472469a101/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:36.695091 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb737b-572d-4f90-b160-98dfa2967fe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d13a5d809e74cea8053c918f6ece4d3, user_id=5e2504d045d247f584d294472469a101, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:36.697718 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb737b-572d-4f90-b160-98dfa2967fe0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:36.710309 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1435/2861] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:43:36 2022] PUT /identity/v3/projects/8d13a5d809e74cea8053c918f6ece4d3/users/5e2504d045d247f584d294472469a101/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:36.720218 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed643001-2874-4764-bd91-4d178ed18be6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:36.720704 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed643001-2874-4764-bd91-4d178ed18be6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:36.720804 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed643001-2874-4764-bd91-4d178ed18be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:36.749078 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ed643001-2874-4764-bd91-4d178ed18be6 None None] MFA Rules not processed for user `5e2504d045d247f584d294472469a101`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:36.750306 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ed643001-2874-4764-bd91-4d178ed18be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:36.811512 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1427/2862] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:36.816211 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb737b-572d-4f90-b160-98dfa2967fe0 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:36.876986 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb737b-572d-4f90-b160-98dfa2967fe0 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:36.881637 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a43ff9-221d-4a7d-9219-b90d85c05f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:36.882164 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a43ff9-221d-4a7d-9219-b90d85c05f71 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:36.882471 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a43ff9-221d-4a7d-9219-b90d85c05f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:36.882747 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a43ff9-221d-4a7d-9219-b90d85c05f71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:36.883359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a43ff9-221d-4a7d-9219-b90d85c05f71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:36.885237 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a43ff9-221d-4a7d-9219-b90d85c05f71 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:36.911507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1436/2863] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:43:36 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:43:36.915493 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed643001-2874-4764-bd91-4d178ed18be6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:36.926001 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed643001-2874-4764-bd91-4d178ed18be6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:36.981599 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331886e-721b-4ca0-a396-ff7fe7fe2f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:36.982180 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331886e-721b-4ca0-a396-ff7fe7fe2f83 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:36.982489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331886e-721b-4ca0-a396-ff7fe7fe2f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:36.982756 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331886e-721b-4ca0-a396-ff7fe7fe2f83 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:36.983374 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1331886e-721b-4ca0-a396-ff7fe7fe2f83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:36.985269 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1331886e-721b-4ca0-a396-ff7fe7fe2f83 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:37.001733 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:37.007138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1428/2864] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:43:36 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:43:37.011038 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a43ff9-221d-4a7d-9219-b90d85c05f71 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:37.021745 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a43ff9-221d-4a7d-9219-b90d85c05f71 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:37.026507 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c147396-e3c3-4ee7-be10-fe6dba3fcd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:37.027097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c147396-e3c3-4ee7-be10-fe6dba3fcd94 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:37.027373 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c147396-e3c3-4ee7-be10-fe6dba3fcd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:37.027638 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c147396-e3c3-4ee7-be10-fe6dba3fcd94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:37.028268 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c147396-e3c3-4ee7-be10-fe6dba3fcd94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:37.029904 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c147396-e3c3-4ee7-be10-fe6dba3fcd94 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:37.040921 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1437/2865] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:37.044878 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331886e-721b-4ca0-a396-ff7fe7fe2f83 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:37.054748 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331886e-721b-4ca0-a396-ff7fe7fe2f83 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:37.059252 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435be77-a8f0-4e98-ba78-e9c28c81e6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:37.059880 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435be77-a8f0-4e98-ba78-e9c28c81e6ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:37.060154 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435be77-a8f0-4e98-ba78-e9c28c81e6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:37.060418 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435be77-a8f0-4e98-ba78-e9c28c81e6ea admin admin] PATH_INFO: `/v3/projects/d3bfa7fef0b54db9a74c4015a28236ce/users/e5b2f7124aee42cfb27b32a983a6fd56/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:37.075130 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435be77-a8f0-4e98-ba78-e9c28c81e6ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3bfa7fef0b54db9a74c4015a28236ce, user_id=e5b2f7124aee42cfb27b32a983a6fd56, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:37.077835 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435be77-a8f0-4e98-ba78-e9c28c81e6ea admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:37.091293 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1429/2866] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:43:37 2022] PUT /identity/v3/projects/d3bfa7fef0b54db9a74c4015a28236ce/users/e5b2f7124aee42cfb27b32a983a6fd56/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:37.095212 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c147396-e3c3-4ee7-be10-fe6dba3fcd94 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:37.105624 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c147396-e3c3-4ee7-be10-fe6dba3fcd94 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:37.142753 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:37.162913 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e06b7-e536-4ec4-b2fe-de1f597d4001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:37.163510 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e06b7-e536-4ec4-b2fe-de1f597d4001 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:37.163785 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e06b7-e536-4ec4-b2fe-de1f597d4001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:37.164056 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e06b7-e536-4ec4-b2fe-de1f597d4001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:37.164690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e06b7-e536-4ec4-b2fe-de1f597d4001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:37.166288 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e06b7-e536-4ec4-b2fe-de1f597d4001 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:37.175800 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1438/2867] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:37.180033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435be77-a8f0-4e98-ba78-e9c28c81e6ea admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:37.191142 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435be77-a8f0-4e98-ba78-e9c28c81e6ea admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:37.195669 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76d980-41b4-468b-bf76-54e4f165bb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:37.196302 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76d980-41b4-468b-bf76-54e4f165bb87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:37.196576 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76d980-41b4-468b-bf76-54e4f165bb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:37.196841 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76d980-41b4-468b-bf76-54e4f165bb87 admin admin] PATH_INFO: `/v3/projects/d3bfa7fef0b54db9a74c4015a28236ce/users/e5b2f7124aee42cfb27b32a983a6fd56/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:37.198807 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76d980-41b4-468b-bf76-54e4f165bb87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3bfa7fef0b54db9a74c4015a28236ce, user_id=e5b2f7124aee42cfb27b32a983a6fd56, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:37.201398 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76d980-41b4-468b-bf76-54e4f165bb87 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:37.211910 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1430/2868] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:43:37 2022] PUT /identity/v3/projects/d3bfa7fef0b54db9a74c4015a28236ce/users/e5b2f7124aee42cfb27b32a983a6fd56/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:37.222895 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b500df4-838a-432d-8064-b7e9807d0d12 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:37.223184 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b500df4-838a-432d-8064-b7e9807d0d12 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:37.223458 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b500df4-838a-432d-8064-b7e9807d0d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:37.251761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8b500df4-838a-432d-8064-b7e9807d0d12 None None] MFA Rules not processed for user `e5b2f7124aee42cfb27b32a983a6fd56`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:37.253516 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8b500df4-838a-432d-8064-b7e9807d0d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:37.319530 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1439/2869] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:37.331994 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76d980-41b4-468b-bf76-54e4f165bb87 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:37.333684 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fd76d980-41b4-468b-bf76-54e4f165bb87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:37.389562 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76d980-41b4-468b-bf76-54e4f165bb87 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:37.393791 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00702b-687a-4741-930e-349613b70628 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:37.394344 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00702b-687a-4741-930e-349613b70628 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:37.394618 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00702b-687a-4741-930e-349613b70628 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:37.394885 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00702b-687a-4741-930e-349613b70628 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:37.449633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00702b-687a-4741-930e-349613b70628 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:37.451095 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00702b-687a-4741-930e-349613b70628 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:37.460392 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1431/2870] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:39.981188 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b500df4-838a-432d-8064-b7e9807d0d12 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:40.041625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b500df4-838a-432d-8064-b7e9807d0d12 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:40.046330 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57684493-5de1-4c78-be0e-02082be57037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:40.046940 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57684493-5de1-4c78-be0e-02082be57037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:40.047216 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57684493-5de1-4c78-be0e-02082be57037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:40.047483 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57684493-5de1-4c78-be0e-02082be57037 admin admin] PATH_INFO: `/v3/users/5e2504d045d247f584d294472469a101` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:40.048579 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57684493-5de1-4c78-be0e-02082be57037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e2504d045d247f584d294472469a101)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:40.050551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57684493-5de1-4c78-be0e-02082be57037 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:40.095256 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-57684493-5de1-4c78-be0e-02082be57037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e2504d045d247f584d294472469a101'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:40.102441 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-57684493-5de1-4c78-be0e-02082be57037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e2504d045d247f584d294472469a101'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:40.109716 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-57684493-5de1-4c78-be0e-02082be57037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e2504d045d247f584d294472469a101'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:40.110451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-57684493-5de1-4c78-be0e-02082be57037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e2504d045d247f584d294472469a101'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:40.111055 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-57684493-5de1-4c78-be0e-02082be57037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e2504d045d247f584d294472469a101'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:40.112184 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:40.131258 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1440/2871] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:43:39 2022] DELETE /identity/v3/users/5e2504d045d247f584d294472469a101 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:40.136067 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30dc0c9-6a24-4fed-bede-980a4596337c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:40.136559 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30dc0c9-6a24-4fed-bede-980a4596337c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:40.136675 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30dc0c9-6a24-4fed-bede-980a4596337c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:40.155499 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e30dc0c9-6a24-4fed-bede-980a4596337c None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:40.156699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e30dc0c9-6a24-4fed-bede-980a4596337c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:40.214332 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1432/2872] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:43:40.223882 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57684493-5de1-4c78-be0e-02082be57037 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:40.225263 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-57684493-5de1-4c78-be0e-02082be57037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:40.282318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57684493-5de1-4c78-be0e-02082be57037 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:40.286108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57087b39-d324-40c3-9730-5c7d0b518253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:40.286757 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57087b39-d324-40c3-9730-5c7d0b518253 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:40.287034 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57087b39-d324-40c3-9730-5c7d0b518253 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:40.287304 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57087b39-d324-40c3-9730-5c7d0b518253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:40.296290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57087b39-d324-40c3-9730-5c7d0b518253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:40.297804 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57087b39-d324-40c3-9730-5c7d0b518253 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:40.307423 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1441/2873] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:40.568141 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30dc0c9-6a24-4fed-bede-980a4596337c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:40.569600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e30dc0c9-6a24-4fed-bede-980a4596337c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:40.584044 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30dc0c9-6a24-4fed-bede-980a4596337c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:40.589403 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27594627-a8bc-4e2b-bbd6-0d98ae7d07f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:40.590112 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27594627-a8bc-4e2b-bbd6-0d98ae7d07f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:40.590337 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27594627-a8bc-4e2b-bbd6-0d98ae7d07f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:40.590610 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27594627-a8bc-4e2b-bbd6-0d98ae7d07f4 admin admin] PATH_INFO: `/v3/projects/8d13a5d809e74cea8053c918f6ece4d3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:40.591745 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27594627-a8bc-4e2b-bbd6-0d98ae7d07f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d13a5d809e74cea8053c918f6ece4d3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:40.593714 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27594627-a8bc-4e2b-bbd6-0d98ae7d07f4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:40.605328 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:40.643367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-27594627-a8bc-4e2b-bbd6-0d98ae7d07f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d13a5d809e74cea8053c918f6ece4d3', 'request_id': 'req-27594627-a8bc-4e2b-bbd6-0d98ae7d07f4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:40.650234 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-27594627-a8bc-4e2b-bbd6-0d98ae7d07f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d13a5d809e74cea8053c918f6ece4d3', 'request_id': 'req-27594627-a8bc-4e2b-bbd6-0d98ae7d07f4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:40.658718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-27594627-a8bc-4e2b-bbd6-0d98ae7d07f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d13a5d809e74cea8053c918f6ece4d3', 'request_id': 'req-27594627-a8bc-4e2b-bbd6-0d98ae7d07f4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:40.670896 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-27594627-a8bc-4e2b-bbd6-0d98ae7d07f4 admin admin] The token cache is being invalidate because project 8d13a5d809e74cea8053c918f6ece4d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:43:40.671183 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-27594627-a8bc-4e2b-bbd6-0d98ae7d07f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:40.671833 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-27594627-a8bc-4e2b-bbd6-0d98ae7d07f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:40.673351 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1433/2874] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:43:40 2022] DELETE /identity/v3/projects/8d13a5d809e74cea8053c918f6ece4d3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:40.677854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57087b39-d324-40c3-9730-5c7d0b518253 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:40.687549 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57087b39-d324-40c3-9730-5c7d0b518253 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:40.743616 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76a8dd-4de2-4603-8710-04c61646115a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:40.744242 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76a8dd-4de2-4603-8710-04c61646115a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:40.744518 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76a8dd-4de2-4603-8710-04c61646115a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:40.744793 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76a8dd-4de2-4603-8710-04c61646115a admin admin] PATH_INFO: `/v3/users/e5b2f7124aee42cfb27b32a983a6fd56` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:40.745914 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76a8dd-4de2-4603-8710-04c61646115a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5b2f7124aee42cfb27b32a983a6fd56)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:40.747856 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76a8dd-4de2-4603-8710-04c61646115a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:40.792352 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bf76a8dd-4de2-4603-8710-04c61646115a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5b2f7124aee42cfb27b32a983a6fd56'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:40.799594 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bf76a8dd-4de2-4603-8710-04c61646115a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5b2f7124aee42cfb27b32a983a6fd56'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:40.807378 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bf76a8dd-4de2-4603-8710-04c61646115a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5b2f7124aee42cfb27b32a983a6fd56'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:40.808139 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bf76a8dd-4de2-4603-8710-04c61646115a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5b2f7124aee42cfb27b32a983a6fd56'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:40.808740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bf76a8dd-4de2-4603-8710-04c61646115a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5b2f7124aee42cfb27b32a983a6fd56'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:40.826318 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1442/2875] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:43:40 2022] DELETE /identity/v3/users/e5b2f7124aee42cfb27b32a983a6fd56 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:41.101073 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27594627-a8bc-4e2b-bbd6-0d98ae7d07f4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:41.102586 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-27594627-a8bc-4e2b-bbd6-0d98ae7d07f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:41.117750 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27594627-a8bc-4e2b-bbd6-0d98ae7d07f4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:41.175145 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee954d-59db-41ad-99cc-f81b75658f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:41.175820 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee954d-59db-41ad-99cc-f81b75658f59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:41.175965 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee954d-59db-41ad-99cc-f81b75658f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:41.176238 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee954d-59db-41ad-99cc-f81b75658f59 admin admin] PATH_INFO: `/v3/projects/d3bfa7fef0b54db9a74c4015a28236ce` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:41.177364 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee954d-59db-41ad-99cc-f81b75658f59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3bfa7fef0b54db9a74c4015a28236ce)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:41.179431 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee954d-59db-41ad-99cc-f81b75658f59 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:41.224039 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8cee954d-59db-41ad-99cc-f81b75658f59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3bfa7fef0b54db9a74c4015a28236ce', 'request_id': 'req-8cee954d-59db-41ad-99cc-f81b75658f59'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:41.229412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8cee954d-59db-41ad-99cc-f81b75658f59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3bfa7fef0b54db9a74c4015a28236ce', 'request_id': 'req-8cee954d-59db-41ad-99cc-f81b75658f59'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:41.237094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8cee954d-59db-41ad-99cc-f81b75658f59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3bfa7fef0b54db9a74c4015a28236ce', 'request_id': 'req-8cee954d-59db-41ad-99cc-f81b75658f59'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:41.249113 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8cee954d-59db-41ad-99cc-f81b75658f59 admin admin] The token cache is being invalidate because project d3bfa7fef0b54db9a74c4015a28236ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:43:41.249467 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8cee954d-59db-41ad-99cc-f81b75658f59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:41.250185 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8cee954d-59db-41ad-99cc-f81b75658f59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:41.251762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1434/2876] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:43:41 2022] DELETE /identity/v3/projects/d3bfa7fef0b54db9a74c4015a28236ce => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:41.256690 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68fb12-f34e-4907-89eb-5fc28466e878 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:41.257174 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68fb12-f34e-4907-89eb-5fc28466e878 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:41.257296 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68fb12-f34e-4907-89eb-5fc28466e878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:41.264976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:41.279436 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-bd68fb12-f34e-4907-89eb-5fc28466e878 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:41.280697 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bd68fb12-f34e-4907-89eb-5fc28466e878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:41.338008 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1443/2877] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:41.346589 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec2fae-04b9-4fe9-8e18-c715a00ac5fd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:41.346882 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec2fae-04b9-4fe9-8e18-c715a00ac5fd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:41.347153 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec2fae-04b9-4fe9-8e18-c715a00ac5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:41.366057 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-47ec2fae-04b9-4fe9-8e18-c715a00ac5fd None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:41.367300 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-47ec2fae-04b9-4fe9-8e18-c715a00ac5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:41.375118 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1435/2878] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:41.379172 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68fb12-f34e-4907-89eb-5fc28466e878 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:41.388739 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68fb12-f34e-4907-89eb-5fc28466e878 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:41.393870 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b2158-067a-48b6-93e9-1be34ca1e3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:41.394490 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b2158-067a-48b6-93e9-1be34ca1e3c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:41.394767 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b2158-067a-48b6-93e9-1be34ca1e3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:41.395037 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b2158-067a-48b6-93e9-1be34ca1e3c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:41.395678 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b2158-067a-48b6-93e9-1be34ca1e3c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:43:41.395982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b2158-067a-48b6-93e9-1be34ca1e3c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:41.397565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b2158-067a-48b6-93e9-1be34ca1e3c5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:41.409457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1444/2879] 10.222.0.39 () {58 vars in 1153 bytes} [Mon Aug 8 02:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:43:41.413600 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec2fae-04b9-4fe9-8e18-c715a00ac5fd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:41.422964 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec2fae-04b9-4fe9-8e18-c715a00ac5fd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:41.427921 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370400f-529e-4cb0-80d6-1e4c23f1a936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:41.428498 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370400f-529e-4cb0-80d6-1e4c23f1a936 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:41.428782 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370400f-529e-4cb0-80d6-1e4c23f1a936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:41.429049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370400f-529e-4cb0-80d6-1e4c23f1a936 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:41.429706 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370400f-529e-4cb0-80d6-1e4c23f1a936 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:41.431647 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370400f-529e-4cb0-80d6-1e4c23f1a936 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:41.459861 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1436/2880] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:43:41 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:43:41.464087 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b2158-067a-48b6-93e9-1be34ca1e3c5 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:41.474686 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b2158-067a-48b6-93e9-1be34ca1e3c5 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:41.531030 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63d15a-5dc3-48f1-9920-a6e7adfea3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:41.531621 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63d15a-5dc3-48f1-9920-a6e7adfea3be admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:41.531896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63d15a-5dc3-48f1-9920-a6e7adfea3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:41.532164 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63d15a-5dc3-48f1-9920-a6e7adfea3be admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:41.532770 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63d15a-5dc3-48f1-9920-a6e7adfea3be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:41.535262 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63d15a-5dc3-48f1-9920-a6e7adfea3be admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:41.555436 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1445/2881] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:43:41 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:43:41.559518 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370400f-529e-4cb0-80d6-1e4c23f1a936 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:41.565115 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:41.571736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370400f-529e-4cb0-80d6-1e4c23f1a936 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:41.577186 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254a308-d0b7-444a-addd-4e0210e612b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:41.577796 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254a308-d0b7-444a-addd-4e0210e612b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:41.578075 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254a308-d0b7-444a-addd-4e0210e612b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:41.578382 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254a308-d0b7-444a-addd-4e0210e612b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:41.579030 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254a308-d0b7-444a-addd-4e0210e612b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:41.581099 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254a308-d0b7-444a-addd-4e0210e612b0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:41.590846 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1437/2882] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:41.594563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63d15a-5dc3-48f1-9920-a6e7adfea3be admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:41.604959 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63d15a-5dc3-48f1-9920-a6e7adfea3be admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:41.609485 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8840a-0bc7-4a03-b74d-49a578d59039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:41.610074 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8840a-0bc7-4a03-b74d-49a578d59039 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:41.610386 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8840a-0bc7-4a03-b74d-49a578d59039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:41.610660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8840a-0bc7-4a03-b74d-49a578d59039 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:41.611294 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8840a-0bc7-4a03-b74d-49a578d59039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:41.612869 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8840a-0bc7-4a03-b74d-49a578d59039 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:41.623290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1446/2883] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:41.627800 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254a308-d0b7-444a-addd-4e0210e612b0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:41.637609 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254a308-d0b7-444a-addd-4e0210e612b0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:41.642573 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbacdb7-8127-434c-8e57-b57a23774668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:41.643793 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbacdb7-8127-434c-8e57-b57a23774668 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:41.644069 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbacdb7-8127-434c-8e57-b57a23774668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:41.644338 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbacdb7-8127-434c-8e57-b57a23774668 admin admin] PATH_INFO: `/v3/projects/94219d4e012248139b9df01bcdf39bbc/users/57bd48dc29f945b9873334293d85b783/roles/42c846f6944b40c8af5eeb8360cb7c7a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:41.658674 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbacdb7-8127-434c-8e57-b57a23774668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94219d4e012248139b9df01bcdf39bbc, user_id=57bd48dc29f945b9873334293d85b783, role_id=42c846f6944b40c8af5eeb8360cb7c7a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:41.661316 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbacdb7-8127-434c-8e57-b57a23774668 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:41.673696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1438/2884] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:43:41 2022] PUT /identity/v3/projects/94219d4e012248139b9df01bcdf39bbc/users/57bd48dc29f945b9873334293d85b783/roles/42c846f6944b40c8af5eeb8360cb7c7a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:41.678756 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2065840-2291-45d2-be39-9bf09a9b7634 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:41.679052 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2065840-2291-45d2-be39-9bf09a9b7634 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:41.679325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2065840-2291-45d2-be39-9bf09a9b7634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:41.698216 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b2065840-2291-45d2-be39-9bf09a9b7634 None None] MFA Rules not processed for user `2d716e3acbb54b2fa8e10b30ba8eb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:41.699440 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b2065840-2291-45d2-be39-9bf09a9b7634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:41.758305 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1447/2885] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:41.768534 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbacdb7-8127-434c-8e57-b57a23774668 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:41.770195 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3fbacdb7-8127-434c-8e57-b57a23774668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:41.825736 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbacdb7-8127-434c-8e57-b57a23774668 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:41.829336 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65772a-f773-4c9a-80c4-38f6bb3765ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:41.829927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65772a-f773-4c9a-80c4-38f6bb3765ec service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:41.830860 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65772a-f773-4c9a-80c4-38f6bb3765ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:41.831138 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65772a-f773-4c9a-80c4-38f6bb3765ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:41.839949 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65772a-f773-4c9a-80c4-38f6bb3765ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:41.841366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65772a-f773-4c9a-80c4-38f6bb3765ec service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:41.850875 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1439/2886] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:45.871982 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba022c8-0817-42b6-83da-5e6879ffa648 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:45.872457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba022c8-0817-42b6-83da-5e6879ffa648 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:45.872562 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba022c8-0817-42b6-83da-5e6879ffa648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:45.900669 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:45.913431 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-1ba022c8-0817-42b6-83da-5e6879ffa648 None None] MFA Rules not processed for user `57bd48dc29f945b9873334293d85b783`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:45.914815 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1ba022c8-0817-42b6-83da-5e6879ffa648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:46.012382 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1448/2887] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:46.016862 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65772a-f773-4c9a-80c4-38f6bb3765ec service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:46.040476 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65772a-f773-4c9a-80c4-38f6bb3765ec service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:46.045116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36914318-b2ae-4aae-b9d6-c1816871b6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:46.045713 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36914318-b2ae-4aae-b9d6-c1816871b6b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:46.045997 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36914318-b2ae-4aae-b9d6-c1816871b6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:46.046295 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36914318-b2ae-4aae-b9d6-c1816871b6b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:46.046934 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36914318-b2ae-4aae-b9d6-c1816871b6b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:46.049367 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36914318-b2ae-4aae-b9d6-c1816871b6b6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:46.081033 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1440/2888] 10.222.0.39 () {60 vars in 1153 bytes} [Mon Aug 8 02:43:46 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:43:46.085220 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba022c8-0817-42b6-83da-5e6879ffa648 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:46.096761 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba022c8-0817-42b6-83da-5e6879ffa648 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:46.154681 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac869c-da7e-4cde-8ebf-11e3eb30991b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:46.155325 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac869c-da7e-4cde-8ebf-11e3eb30991b admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:46.155607 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac869c-da7e-4cde-8ebf-11e3eb30991b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:46.155878 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac869c-da7e-4cde-8ebf-11e3eb30991b admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:46.156505 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac869c-da7e-4cde-8ebf-11e3eb30991b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:46.158976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac869c-da7e-4cde-8ebf-11e3eb30991b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:46.184780 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1449/2889] 10.222.0.39 () {60 vars in 1144 bytes} [Mon Aug 8 02:43:46 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:43:46.187472 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36914318-b2ae-4aae-b9d6-c1816871b6b6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:46.199225 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36914318-b2ae-4aae-b9d6-c1816871b6b6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:46.204363 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26996ed2-eb39-4217-acfa-9a2b296a1772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:46.204984 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26996ed2-eb39-4217-acfa-9a2b296a1772 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:46.205313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26996ed2-eb39-4217-acfa-9a2b296a1772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:46.205593 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26996ed2-eb39-4217-acfa-9a2b296a1772 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:46.206225 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26996ed2-eb39-4217-acfa-9a2b296a1772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:46.207930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26996ed2-eb39-4217-acfa-9a2b296a1772 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:46.218478 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1441/2890] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:46.223086 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac869c-da7e-4cde-8ebf-11e3eb30991b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:46.232948 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac869c-da7e-4cde-8ebf-11e3eb30991b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:46.238340 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114f6bd-a421-40bd-91ad-1b761d943efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:46.238969 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114f6bd-a421-40bd-91ad-1b761d943efb admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:46.239246 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114f6bd-a421-40bd-91ad-1b761d943efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:46.239551 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114f6bd-a421-40bd-91ad-1b761d943efb admin admin] PATH_INFO: `/v3/projects/1d68ad9b87794bb89b098a97c04ce073/users/a4298d7435b4442fb88f1a823dc3fc35/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:46.256627 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114f6bd-a421-40bd-91ad-1b761d943efb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d68ad9b87794bb89b098a97c04ce073, user_id=a4298d7435b4442fb88f1a823dc3fc35, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:46.259758 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114f6bd-a421-40bd-91ad-1b761d943efb admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:46.276740 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1450/2891] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:43:46 2022] PUT /identity/v3/projects/1d68ad9b87794bb89b098a97c04ce073/users/a4298d7435b4442fb88f1a823dc3fc35/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:43:46.280578 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26996ed2-eb39-4217-acfa-9a2b296a1772 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:46.291134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26996ed2-eb39-4217-acfa-9a2b296a1772 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:46.349746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f5030-2d8d-4601-8cb0-09e19f140d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:46.350425 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f5030-2d8d-4601-8cb0-09e19f140d87 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:46.350699 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f5030-2d8d-4601-8cb0-09e19f140d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:46.351029 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f5030-2d8d-4601-8cb0-09e19f140d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:46.351671 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f5030-2d8d-4601-8cb0-09e19f140d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:46.353313 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f5030-2d8d-4601-8cb0-09e19f140d87 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:46.363966 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1442/2892] 10.222.0.39 () {58 vars in 1122 bytes} [Mon Aug 8 02:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:43:46.368125 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114f6bd-a421-40bd-91ad-1b761d943efb admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:46.378909 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114f6bd-a421-40bd-91ad-1b761d943efb admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:46.383705 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66ed41-15de-4b79-ad00-312bd3e91d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:46.384369 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66ed41-15de-4b79-ad00-312bd3e91d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:46.384660 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66ed41-15de-4b79-ad00-312bd3e91d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:46.384917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66ed41-15de-4b79-ad00-312bd3e91d6b admin admin] PATH_INFO: `/v3/projects/1d68ad9b87794bb89b098a97c04ce073/users/a4298d7435b4442fb88f1a823dc3fc35/roles/ebae0c49752b42c7b7b72945e76d9fe3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:46.386906 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66ed41-15de-4b79-ad00-312bd3e91d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d68ad9b87794bb89b098a97c04ce073, user_id=a4298d7435b4442fb88f1a823dc3fc35, role_id=ebae0c49752b42c7b7b72945e76d9fe3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:46.389553 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66ed41-15de-4b79-ad00-312bd3e91d6b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:46.401593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1451/2893] 10.222.0.39 () {60 vars in 1483 bytes} [Mon Aug 8 02:43:46 2022] PUT /identity/v3/projects/1d68ad9b87794bb89b098a97c04ce073/users/a4298d7435b4442fb88f1a823dc3fc35/roles/ebae0c49752b42c7b7b72945e76d9fe3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:49.837489 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f5030-2d8d-4601-8cb0-09e19f140d87 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:49.839256 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-454f5030-2d8d-4601-8cb0-09e19f140d87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:49.879080 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:49.951960 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f5030-2d8d-4601-8cb0-09e19f140d87 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:50.057998 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5927a5b-3e14-4054-a04a-b1be32df09ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2812e59d0d9948879a465fe3b5b421e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:50.059261 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5927a5b-3e14-4054-a04a-b1be32df09ec service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:50.059761 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5927a5b-3e14-4054-a04a-b1be32df09ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:50.060245 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5927a5b-3e14-4054-a04a-b1be32df09ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:50.062492 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a5927a5b-3e14-4054-a04a-b1be32df09ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:50.176192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5927a5b-3e14-4054-a04a-b1be32df09ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:50.178285 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5927a5b-3e14-4054-a04a-b1be32df09ec service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:50.195999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1443/2894] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:51.009282 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945057c-f614-4d0c-ab5f-c705e93d47dc None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:51.009741 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945057c-f614-4d0c-ab5f-c705e93d47dc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:51.009854 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945057c-f614-4d0c-ab5f-c705e93d47dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:51.040102 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-3945057c-f614-4d0c-ab5f-c705e93d47dc None None] MFA Rules not processed for user `a4298d7435b4442fb88f1a823dc3fc35`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:51.041529 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3945057c-f614-4d0c-ab5f-c705e93d47dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:51.072316 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:51.135005 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1452/2895] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:51.144135 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5927a5b-3e14-4054-a04a-b1be32df09ec service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:51.145693 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a5927a5b-3e14-4054-a04a-b1be32df09ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:51.226552 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5927a5b-3e14-4054-a04a-b1be32df09ec service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:51.231930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f8bb4-00a5-4a05-873c-692eaa9267fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:51.232524 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f8bb4-00a5-4a05-873c-692eaa9267fb service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:51.232799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f8bb4-00a5-4a05-873c-692eaa9267fb service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:51.233064 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f8bb4-00a5-4a05-873c-692eaa9267fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:51.260693 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:51.292404 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f8bb4-00a5-4a05-873c-692eaa9267fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:51.293986 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f8bb4-00a5-4a05-873c-692eaa9267fb service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:51.302865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1444/2896] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:52.314223 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885bd1eb-4b02-46fa-88ad-226989037364 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:52.314766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885bd1eb-4b02-46fa-88ad-226989037364 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:52.314950 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885bd1eb-4b02-46fa-88ad-226989037364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:52.333900 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-885bd1eb-4b02-46fa-88ad-226989037364 None None] MFA Rules not processed for user `57bd48dc29f945b9873334293d85b783`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:52.335172 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-885bd1eb-4b02-46fa-88ad-226989037364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:52.342368 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1453/2897] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:43:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:01.298767 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=15 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:02.328565 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=20 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:02.556111 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16972be-49ae-4bac-acda-0b4c2177ff88 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:02.556633 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16972be-49ae-4bac-acda-0b4c2177ff88 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:02.556799 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16972be-49ae-4bac-acda-0b4c2177ff88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:02.585917 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c16972be-49ae-4bac-acda-0b4c2177ff88 None None] MFA Rules not processed for user `72485bc9ed6f4684b77b617597f99e29`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:02.587163 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c16972be-49ae-4bac-acda-0b4c2177ff88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:02.597777 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1445/2898] 10.222.0.39 () {58 vars in 958 bytes} [Mon Aug 8 02:44:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:02.921243 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885bd1eb-4b02-46fa-88ad-226989037364 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:03.006930 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885bd1eb-4b02-46fa-88ad-226989037364 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:03.010567 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aeeaa0-3eda-4802-95b7-260613265f9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:03.011165 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aeeaa0-3eda-4802-95b7-260613265f9c service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:03.011449 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aeeaa0-3eda-4802-95b7-260613265f9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:03.011722 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aeeaa0-3eda-4802-95b7-260613265f9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:03.013467 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-23aeeaa0-3eda-4802-95b7-260613265f9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:03.038204 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aeeaa0-3eda-4802-95b7-260613265f9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:03.039668 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aeeaa0-3eda-4802-95b7-260613265f9c service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:03.047593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1454/2899] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:04.508615 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16972be-49ae-4bac-acda-0b4c2177ff88 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:04.510022 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c16972be-49ae-4bac-acda-0b4c2177ff88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:04.589092 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16972be-49ae-4bac-acda-0b4c2177ff88 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:04.593942 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f51cd-330c-403b-8561-a99acc8d9d16 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f62745b92a864480a9438de581e1d05e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:04.594570 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f51cd-330c-403b-8561-a99acc8d9d16 service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:04.594845 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f51cd-330c-403b-8561-a99acc8d9d16 service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:04.595112 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f51cd-330c-403b-8561-a99acc8d9d16 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:04.596778 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3b4f51cd-330c-403b-8561-a99acc8d9d16 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:04.608916 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f51cd-330c-403b-8561-a99acc8d9d16 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:04.610085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f51cd-330c-403b-8561-a99acc8d9d16 service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:04.620857 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1446/2900] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:13.044022 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=44 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:14.616491 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=44 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:23.570443 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aeeaa0-3eda-4802-95b7-260613265f9c service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:23.579026 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aeeaa0-3eda-4802-95b7-260613265f9c service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:23.582654 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051418a-29dd-4f12-82e9-cef79866aa3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:23.583215 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051418a-29dd-4f12-82e9-cef79866aa3d service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:23.583499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051418a-29dd-4f12-82e9-cef79866aa3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:23.583771 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051418a-29dd-4f12-82e9-cef79866aa3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:23.585466 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8051418a-29dd-4f12-82e9-cef79866aa3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:23.647385 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051418a-29dd-4f12-82e9-cef79866aa3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:23.648820 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051418a-29dd-4f12-82e9-cef79866aa3d service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:23.658600 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1455/2901] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:33.654685 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=30 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:33.709008 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f51cd-330c-403b-8561-a99acc8d9d16 service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:33.718239 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f51cd-330c-403b-8561-a99acc8d9d16 service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:33.721770 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd92b7f-ec7d-4be3-bc52-cb24018f2a1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:33.722387 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd92b7f-ec7d-4be3-bc52-cb24018f2a1c service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:33.722663 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd92b7f-ec7d-4be3-bc52-cb24018f2a1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:33.722927 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd92b7f-ec7d-4be3-bc52-cb24018f2a1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:33.732014 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd92b7f-ec7d-4be3-bc52-cb24018f2a1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:33.733447 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd92b7f-ec7d-4be3-bc52-cb24018f2a1c service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:33.743059 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1447/2902] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:43.739427 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:47.149336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051418a-29dd-4f12-82e9-cef79866aa3d service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:47.170963 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051418a-29dd-4f12-82e9-cef79866aa3d service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:47.176501 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba92f6-b192-482d-bac1-9db505c31d80 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f62745b92a864480a9438de581e1d05e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:47.177087 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba92f6-b192-482d-bac1-9db505c31d80 service placement] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:47.177359 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba92f6-b192-482d-bac1-9db505c31d80 service placement] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:47.177625 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba92f6-b192-482d-bac1-9db505c31d80 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:47.179340 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4cba92f6-b192-482d-bac1-9db505c31d80 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:47.203046 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba92f6-b192-482d-bac1-9db505c31d80 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:47.204603 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba92f6-b192-482d-bac1-9db505c31d80 service placement] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:47.214383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1456/2903] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:57.210120 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=15 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:46:16.080402 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd92b7f-ec7d-4be3-bc52-cb24018f2a1c service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:46:16.082807 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8cd92b7f-ec7d-4be3-bc52-cb24018f2a1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:46:16.093876 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd92b7f-ec7d-4be3-bc52-cb24018f2a1c service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:46:16.097996 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa877f-7138-47ad-8a77-f54fe3ffaa19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:46:16.098624 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa877f-7138-47ad-8a77-f54fe3ffaa19 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:46:16.098899 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa877f-7138-47ad-8a77-f54fe3ffaa19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:46:16.099166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa877f-7138-47ad-8a77-f54fe3ffaa19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:46:16.101005 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-86fa877f-7138-47ad-8a77-f54fe3ffaa19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:46:16.136543 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa877f-7138-47ad-8a77-f54fe3ffaa19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:46:16.137986 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa877f-7138-47ad-8a77-f54fe3ffaa19 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:46:16.158567 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1448/2904] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:46:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:46:17.339179 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba92f6-b192-482d-bac1-9db505c31d80 service placement] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:46:17.349514 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba92f6-b192-482d-bac1-9db505c31d80 service placement] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:46:17.353242 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c53d7f-ce97-4ab9-83e9-e10a5178f623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:46:17.353861 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c53d7f-ce97-4ab9-83e9-e10a5178f623 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:46:17.354136 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c53d7f-ce97-4ab9-83e9-e10a5178f623 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:46:17.354457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c53d7f-ce97-4ab9-83e9-e10a5178f623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:46:17.356257 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f1c53d7f-ce97-4ab9-83e9-e10a5178f623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:46:17.370593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c53d7f-ce97-4ab9-83e9-e10a5178f623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:46:17.371941 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c53d7f-ce97-4ab9-83e9-e10a5178f623 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:46:17.383593 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1457/2905] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:46:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:46:26.155762 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=18 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:46:27.378536 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=6 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:46:28.225300 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa877f-7138-47ad-8a77-f54fe3ffaa19 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:46:28.234167 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa877f-7138-47ad-8a77-f54fe3ffaa19 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:46:28.237829 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44bff6-2935-4b2b-b5c2-97ebbc19d182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:46:28.238454 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44bff6-2935-4b2b-b5c2-97ebbc19d182 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:46:28.238728 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44bff6-2935-4b2b-b5c2-97ebbc19d182 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:46:28.238992 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44bff6-2935-4b2b-b5c2-97ebbc19d182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:46:28.240718 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8b44bff6-2935-4b2b-b5c2-97ebbc19d182 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:46:28.325231 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44bff6-2935-4b2b-b5c2-97ebbc19d182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:46:28.326709 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44bff6-2935-4b2b-b5c2-97ebbc19d182 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:46:28.345498 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1449/2906] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:46:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:46:31.367586 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c53d7f-ce97-4ab9-83e9-e10a5178f623 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:46:31.380071 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c53d7f-ce97-4ab9-83e9-e10a5178f623 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:46:31.386020 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e73d3-f3a1-4ec6-89fd-06436c84e700 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f62745b92a864480a9438de581e1d05e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:46:31.386499 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e73d3-f3a1-4ec6-89fd-06436c84e700 service placement] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:46:31.386729 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e73d3-f3a1-4ec6-89fd-06436c84e700 service placement] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:46:31.387118 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e73d3-f3a1-4ec6-89fd-06436c84e700 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:46:31.388734 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1c2e73d3-f3a1-4ec6-89fd-06436c84e700 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:46:31.401867 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e73d3-f3a1-4ec6-89fd-06436c84e700 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:46:31.403506 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e73d3-f3a1-4ec6-89fd-06436c84e700 service placement] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:46:31.415531 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1458/2907] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:46:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:46:34.584988 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44bff6-2935-4b2b-b5c2-97ebbc19d182 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:46:34.584988 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8b44bff6-2935-4b2b-b5c2-97ebbc19d182 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:46:34.593166 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44bff6-2935-4b2b-b5c2-97ebbc19d182 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:46:34.597907 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e04b8-3057-47d4-87b2-77cd90d92412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:46:34.598571 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e04b8-3057-47d4-87b2-77cd90d92412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:46:34.598847 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e04b8-3057-47d4-87b2-77cd90d92412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:46:34.599116 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e04b8-3057-47d4-87b2-77cd90d92412 admin admin] PATH_INFO: `/v3/users/72485bc9ed6f4684b77b617597f99e29` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:46:34.600212 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e04b8-3057-47d4-87b2-77cd90d92412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72485bc9ed6f4684b77b617597f99e29)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:46:34.602110 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e04b8-3057-47d4-87b2-77cd90d92412 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:46:34.653320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6b4e04b8-3057-47d4-87b2-77cd90d92412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72485bc9ed6f4684b77b617597f99e29'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:34.661759 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6b4e04b8-3057-47d4-87b2-77cd90d92412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72485bc9ed6f4684b77b617597f99e29'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:34.668999 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6b4e04b8-3057-47d4-87b2-77cd90d92412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72485bc9ed6f4684b77b617597f99e29'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:34.669639 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6b4e04b8-3057-47d4-87b2-77cd90d92412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72485bc9ed6f4684b77b617597f99e29'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:34.670268 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6b4e04b8-3057-47d4-87b2-77cd90d92412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72485bc9ed6f4684b77b617597f99e29'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:34.687221 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1450/2908] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:46:34 2022] DELETE /identity/v3/users/72485bc9ed6f4684b77b617597f99e29 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:46:34.857760 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e73d3-f3a1-4ec6-89fd-06436c84e700 service placement] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:46:34.870558 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e73d3-f3a1-4ec6-89fd-06436c84e700 service placement] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:46:34.927746 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9380d-3dc6-4ac5-bde6-0554d1416b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:46:34.928362 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9380d-3dc6-4ac5-bde6-0554d1416b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:46:34.928637 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9380d-3dc6-4ac5-bde6-0554d1416b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:46:34.928902 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9380d-3dc6-4ac5-bde6-0554d1416b6c admin admin] PATH_INFO: `/v3/projects/7a06a8947bd84ac398bf498d8816717c` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:46:34.929987 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9380d-3dc6-4ac5-bde6-0554d1416b6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a06a8947bd84ac398bf498d8816717c)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:46:34.931896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9380d-3dc6-4ac5-bde6-0554d1416b6c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:46:34.986056 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-87c9380d-3dc6-4ac5-bde6-0554d1416b6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a06a8947bd84ac398bf498d8816717c', 'request_id': 'req-87c9380d-3dc6-4ac5-bde6-0554d1416b6c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:34.994272 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-87c9380d-3dc6-4ac5-bde6-0554d1416b6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a06a8947bd84ac398bf498d8816717c', 'request_id': 'req-87c9380d-3dc6-4ac5-bde6-0554d1416b6c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:35.003862 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-87c9380d-3dc6-4ac5-bde6-0554d1416b6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a06a8947bd84ac398bf498d8816717c', 'request_id': 'req-87c9380d-3dc6-4ac5-bde6-0554d1416b6c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:35.019059 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-87c9380d-3dc6-4ac5-bde6-0554d1416b6c admin admin] The token cache is being invalidate because project 7a06a8947bd84ac398bf498d8816717c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:46:35.019399 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-87c9380d-3dc6-4ac5-bde6-0554d1416b6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:35.020047 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-87c9380d-3dc6-4ac5-bde6-0554d1416b6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:35.021513 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1459/2909] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:46:34 2022] DELETE /identity/v3/projects/7a06a8947bd84ac398bf498d8816717c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:46:35.025527 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e04b8-3057-47d4-87b2-77cd90d92412 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:46:35.026930 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6b4e04b8-3057-47d4-87b2-77cd90d92412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:46:35.039769 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e04b8-3057-47d4-87b2-77cd90d92412 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:46:35.073003 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:46:35.097010 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c4738-9c64-49db-9189-095df0d13c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:46:35.097616 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c4738-9c64-49db-9189-095df0d13c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:46:35.097888 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c4738-9c64-49db-9189-095df0d13c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:46:35.098154 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c4738-9c64-49db-9189-095df0d13c6f admin admin] PATH_INFO: `/v3/users/aa0cdb6397a94de29ab7434fad1861d9` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:46:35.099299 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c4738-9c64-49db-9189-095df0d13c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa0cdb6397a94de29ab7434fad1861d9)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:46:35.101228 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c4738-9c64-49db-9189-095df0d13c6f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:46:35.145584 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-612c4738-9c64-49db-9189-095df0d13c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa0cdb6397a94de29ab7434fad1861d9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:35.153372 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-612c4738-9c64-49db-9189-095df0d13c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa0cdb6397a94de29ab7434fad1861d9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:35.161026 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-612c4738-9c64-49db-9189-095df0d13c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa0cdb6397a94de29ab7434fad1861d9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:35.161656 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-612c4738-9c64-49db-9189-095df0d13c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa0cdb6397a94de29ab7434fad1861d9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:35.162268 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-612c4738-9c64-49db-9189-095df0d13c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa0cdb6397a94de29ab7434fad1861d9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:35.179485 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1451/2910] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:46:35 2022] DELETE /identity/v3/users/aa0cdb6397a94de29ab7434fad1861d9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:46:35.357954 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9380d-3dc6-4ac5-bde6-0554d1416b6c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:46:35.359563 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-87c9380d-3dc6-4ac5-bde6-0554d1416b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:46:35.386700 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9380d-3dc6-4ac5-bde6-0554d1416b6c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:46:35.452088 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973797f-f336-49b3-bbd1-6ba3a3984baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:46:35.452717 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973797f-f336-49b3-bbd1-6ba3a3984baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:46:35.452994 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973797f-f336-49b3-bbd1-6ba3a3984baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:46:35.453290 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973797f-f336-49b3-bbd1-6ba3a3984baa admin admin] PATH_INFO: `/v3/projects/9bc2e67053b347ea948ecd2b5d497d02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:46:35.454451 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973797f-f336-49b3-bbd1-6ba3a3984baa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bc2e67053b347ea948ecd2b5d497d02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:46:35.456602 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973797f-f336-49b3-bbd1-6ba3a3984baa admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:46:35.478872 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:46:35.506574 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c973797f-f336-49b3-bbd1-6ba3a3984baa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bc2e67053b347ea948ecd2b5d497d02', 'request_id': 'req-c973797f-f336-49b3-bbd1-6ba3a3984baa'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:35.512321 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c973797f-f336-49b3-bbd1-6ba3a3984baa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bc2e67053b347ea948ecd2b5d497d02', 'request_id': 'req-c973797f-f336-49b3-bbd1-6ba3a3984baa'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:35.519959 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c973797f-f336-49b3-bbd1-6ba3a3984baa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bc2e67053b347ea948ecd2b5d497d02', 'request_id': 'req-c973797f-f336-49b3-bbd1-6ba3a3984baa'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:35.532302 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c973797f-f336-49b3-bbd1-6ba3a3984baa admin admin] The token cache is being invalidate because project 9bc2e67053b347ea948ecd2b5d497d02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:46:35.532635 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c973797f-f336-49b3-bbd1-6ba3a3984baa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:35.533373 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c973797f-f336-49b3-bbd1-6ba3a3984baa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:35.535596 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1460/2911] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:46:35 2022] DELETE /identity/v3/projects/9bc2e67053b347ea948ecd2b5d497d02 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:46:45.176192 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:46:45.529150 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:46:46.861972 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c4738-9c64-49db-9189-095df0d13c6f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:46:46.863450 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-612c4738-9c64-49db-9189-095df0d13c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:46:46.919105 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c4738-9c64-49db-9189-095df0d13c6f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:46:46.965794 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeef315-05dd-459e-9376-b82f411899fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:46:46.966441 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeef315-05dd-459e-9376-b82f411899fd service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:46:46.966715 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeef315-05dd-459e-9376-b82f411899fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:46:46.967007 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeef315-05dd-459e-9376-b82f411899fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:46:46.968688 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6eeef315-05dd-459e-9376-b82f411899fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:46:46.979230 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeef315-05dd-459e-9376-b82f411899fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:46:46.980638 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeef315-05dd-459e-9376-b82f411899fd service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:46:46.988982 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1452/2912] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:46:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:46:56.984953 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=46 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:47:06.695276 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973797f-f336-49b3-bbd1-6ba3a3984baa admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:47:06.696802 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c973797f-f336-49b3-bbd1-6ba3a3984baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:47:06.709851 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973797f-f336-49b3-bbd1-6ba3a3984baa admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:47:06.713412 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06588d-5c82-47b6-b8b8-48fefcbebb33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:47:06.714013 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06588d-5c82-47b6-b8b8-48fefcbebb33 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:47:06.714321 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06588d-5c82-47b6-b8b8-48fefcbebb33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:47:06.714598 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06588d-5c82-47b6-b8b8-48fefcbebb33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:47:06.716338 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9f06588d-5c82-47b6-b8b8-48fefcbebb33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:47:06.726069 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06588d-5c82-47b6-b8b8-48fefcbebb33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:47:06.727528 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06588d-5c82-47b6-b8b8-48fefcbebb33 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:47:06.735948 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1461/2913] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:47:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:47:07.405787 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeef315-05dd-459e-9376-b82f411899fd service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:47:07.407396 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6eeef315-05dd-459e-9376-b82f411899fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:47:07.483224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeef315-05dd-459e-9376-b82f411899fd service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:47:07.489199 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e70d6-4f30-4c55-928d-598dd7188e8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:47:07.489806 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e70d6-4f30-4c55-928d-598dd7188e8b service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:47:07.490063 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e70d6-4f30-4c55-928d-598dd7188e8b service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:47:07.490358 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e70d6-4f30-4c55-928d-598dd7188e8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:47:07.492070 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-405e70d6-4f30-4c55-928d-598dd7188e8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:47:07.503827 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e70d6-4f30-4c55-928d-598dd7188e8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:47:07.505394 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e70d6-4f30-4c55-928d-598dd7188e8b service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:47:07.515732 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1453/2914] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:47:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:47:07.816548 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06588d-5c82-47b6-b8b8-48fefcbebb33 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:47:07.818010 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9f06588d-5c82-47b6-b8b8-48fefcbebb33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:47:07.828965 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06588d-5c82-47b6-b8b8-48fefcbebb33 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:47:07.833108 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de43d2-9842-4672-85d1-8834aeff97d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:47:07.833696 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de43d2-9842-4672-85d1-8834aeff97d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:47:07.833976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de43d2-9842-4672-85d1-8834aeff97d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:47:07.834328 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de43d2-9842-4672-85d1-8834aeff97d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:47:07.835938 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-74de43d2-9842-4672-85d1-8834aeff97d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:47:07.849653 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de43d2-9842-4672-85d1-8834aeff97d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:47:07.851105 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de43d2-9842-4672-85d1-8834aeff97d7 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:47:07.862402 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1462/2915] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:47:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:47:17.511321 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=34 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:47:17.857923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=12 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:48:59.241641 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e70d6-4f30-4c55-928d-598dd7188e8b service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:48:59.251094 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e70d6-4f30-4c55-928d-598dd7188e8b service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:48:59.255085 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66050585-a3e1-4b73-bc83-bcf8bb5d10ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:48:59.255666 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66050585-a3e1-4b73-bc83-bcf8bb5d10ab service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:48:59.255940 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66050585-a3e1-4b73-bc83-bcf8bb5d10ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:48:59.256206 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66050585-a3e1-4b73-bc83-bcf8bb5d10ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:48:59.257877 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-66050585-a3e1-4b73-bc83-bcf8bb5d10ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:48:59.313922 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66050585-a3e1-4b73-bc83-bcf8bb5d10ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:48:59.316125 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66050585-a3e1-4b73-bc83-bcf8bb5d10ab service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:48:59.324819 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1454/2916] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:48:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:49:00.428793 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de43d2-9842-4672-85d1-8834aeff97d7 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:49:00.441527 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de43d2-9842-4672-85d1-8834aeff97d7 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:49:00.447930 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e190897-b27f-486b-9a8c-b695817f9886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:49:00.448547 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e190897-b27f-486b-9a8c-b695817f9886 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:49:00.448825 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e190897-b27f-486b-9a8c-b695817f9886 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:49:00.449097 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e190897-b27f-486b-9a8c-b695817f9886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:49:00.450874 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2e190897-b27f-486b-9a8c-b695817f9886 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:49:00.464360 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e190897-b27f-486b-9a8c-b695817f9886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:49:00.465954 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e190897-b27f-486b-9a8c-b695817f9886 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:49:00.478575 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1463/2917] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:49:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:49:02.928122 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66050585-a3e1-4b73-bc83-bcf8bb5d10ab service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:49:02.929919 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-66050585-a3e1-4b73-bc83-bcf8bb5d10ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:49:02.947696 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66050585-a3e1-4b73-bc83-bcf8bb5d10ab service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:49:02.953865 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e2ac8-b343-4747-a3b8-5488510a5491 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4238566a9c248ffbfa314f47f02670c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:49:02.954759 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e2ac8-b343-4747-a3b8-5488510a5491 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:49:02.955134 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e2ac8-b343-4747-a3b8-5488510a5491 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:49:02.955417 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e2ac8-b343-4747-a3b8-5488510a5491 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:49:02.958096 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b41e2ac8-b343-4747-a3b8-5488510a5491 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:49:02.971753 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e2ac8-b343-4747-a3b8-5488510a5491 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:49:02.973366 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e2ac8-b343-4747-a3b8-5488510a5491 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:49:02.988974 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1455/2918] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:49:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:49:04.269896 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e190897-b27f-486b-9a8c-b695817f9886 service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:49:04.279657 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e190897-b27f-486b-9a8c-b695817f9886 service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:49:04.283254 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d14aa-40c4-4d2b-9140-d14cfa745efb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78506fea1c024b169a4b7ee86116658c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:49:04.283836 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d14aa-40c4-4d2b-9140-d14cfa745efb service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:49:04.284124 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d14aa-40c4-4d2b-9140-d14cfa745efb service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:49:04.284404 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d14aa-40c4-4d2b-9140-d14cfa745efb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:49:04.286094 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-091d14aa-40c4-4d2b-9140-d14cfa745efb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:49:04.348712 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d14aa-40c4-4d2b-9140-d14cfa745efb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:49:04.350140 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d14aa-40c4-4d2b-9140-d14cfa745efb service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:49:04.363373 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1464/2919] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:49:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:49:07.404729 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e2ac8-b343-4747-a3b8-5488510a5491 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:49:07.406164 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b41e2ac8-b343-4747-a3b8-5488510a5491 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:49:07.492702 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e2ac8-b343-4747-a3b8-5488510a5491 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:49:07.497655 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5356a3-6afb-4f92-8058-abb97c9f70c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f62745b92a864480a9438de581e1d05e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:49:07.498278 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5356a3-6afb-4f92-8058-abb97c9f70c9 service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:49:07.498558 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5356a3-6afb-4f92-8058-abb97c9f70c9 service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:49:07.498828 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5356a3-6afb-4f92-8058-abb97c9f70c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:49:07.500554 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2b5356a3-6afb-4f92-8058-abb97c9f70c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:49:07.512667 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5356a3-6afb-4f92-8058-abb97c9f70c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:49:07.514262 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5356a3-6afb-4f92-8058-abb97c9f70c9 service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:49:07.530722 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1456/2920] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:49:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:49:10.381457 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d14aa-40c4-4d2b-9140-d14cfa745efb service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:49:10.383058 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-091d14aa-40c4-4d2b-9140-d14cfa745efb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:49:10.397363 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d14aa-40c4-4d2b-9140-d14cfa745efb service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:49:10.402100 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72f19d-bfbc-4701-a558-1bf7666c77ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:49:10.404923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72f19d-bfbc-4701-a558-1bf7666c77ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:49:10.404923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72f19d-bfbc-4701-a558-1bf7666c77ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:49:10.404923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72f19d-bfbc-4701-a558-1bf7666c77ba admin admin] PATH_INFO: `/v3/users/57bd48dc29f945b9873334293d85b783` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:49:10.404923 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce72f19d-bfbc-4701-a558-1bf7666c77ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=57bd48dc29f945b9873334293d85b783)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:49:10.406449 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce72f19d-bfbc-4701-a558-1bf7666c77ba admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:49:10.469291 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ce72f19d-bfbc-4701-a558-1bf7666c77ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57bd48dc29f945b9873334293d85b783'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:10.476361 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ce72f19d-bfbc-4701-a558-1bf7666c77ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57bd48dc29f945b9873334293d85b783'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:10.483773 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ce72f19d-bfbc-4701-a558-1bf7666c77ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57bd48dc29f945b9873334293d85b783'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:10.484535 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ce72f19d-bfbc-4701-a558-1bf7666c77ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57bd48dc29f945b9873334293d85b783'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:10.485174 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ce72f19d-bfbc-4701-a558-1bf7666c77ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57bd48dc29f945b9873334293d85b783'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:10.503482 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1465/2921] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:49:10 2022] DELETE /identity/v3/users/57bd48dc29f945b9873334293d85b783 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:49:10.678785 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5356a3-6afb-4f92-8058-abb97c9f70c9 service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:49:10.693746 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5356a3-6afb-4f92-8058-abb97c9f70c9 service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:49:10.748604 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33f8e0-1976-4426-b11a-d338b50d9a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:49:10.749224 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33f8e0-1976-4426-b11a-d338b50d9a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:49:10.749499 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33f8e0-1976-4426-b11a-d338b50d9a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:49:10.749766 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33f8e0-1976-4426-b11a-d338b50d9a99 admin admin] PATH_INFO: `/v3/projects/94219d4e012248139b9df01bcdf39bbc` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:49:10.750916 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33f8e0-1976-4426-b11a-d338b50d9a99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94219d4e012248139b9df01bcdf39bbc)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:49:10.753428 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33f8e0-1976-4426-b11a-d338b50d9a99 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:49:10.770083 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:49:10.799300 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8d33f8e0-1976-4426-b11a-d338b50d9a99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94219d4e012248139b9df01bcdf39bbc', 'request_id': 'req-8d33f8e0-1976-4426-b11a-d338b50d9a99'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:10.805118 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8d33f8e0-1976-4426-b11a-d338b50d9a99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94219d4e012248139b9df01bcdf39bbc', 'request_id': 'req-8d33f8e0-1976-4426-b11a-d338b50d9a99'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:10.812405 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8d33f8e0-1976-4426-b11a-d338b50d9a99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94219d4e012248139b9df01bcdf39bbc', 'request_id': 'req-8d33f8e0-1976-4426-b11a-d338b50d9a99'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:10.824009 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8d33f8e0-1976-4426-b11a-d338b50d9a99 admin admin] The token cache is being invalidate because project 94219d4e012248139b9df01bcdf39bbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:49:10.824299 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8d33f8e0-1976-4426-b11a-d338b50d9a99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:10.824933 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8d33f8e0-1976-4426-b11a-d338b50d9a99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:10.826430 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1457/2922] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:49:10 2022] DELETE /identity/v3/projects/94219d4e012248139b9df01bcdf39bbc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:49:10.830422 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72f19d-bfbc-4701-a558-1bf7666c77ba admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:49:10.831826 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ce72f19d-bfbc-4701-a558-1bf7666c77ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:49:10.845422 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72f19d-bfbc-4701-a558-1bf7666c77ba admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:49:10.891976 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:49:10.902766 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafafd5a-a5e4-4c9c-b086-ce489d504f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:49:10.903383 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafafd5a-a5e4-4c9c-b086-ce489d504f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:49:10.903658 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafafd5a-a5e4-4c9c-b086-ce489d504f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:49:10.903931 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafafd5a-a5e4-4c9c-b086-ce489d504f88 admin admin] PATH_INFO: `/v3/users/a4298d7435b4442fb88f1a823dc3fc35` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:49:10.905366 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafafd5a-a5e4-4c9c-b086-ce489d504f88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4298d7435b4442fb88f1a823dc3fc35)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:49:10.907336 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafafd5a-a5e4-4c9c-b086-ce489d504f88 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:49:10.952889 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-eafafd5a-a5e4-4c9c-b086-ce489d504f88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4298d7435b4442fb88f1a823dc3fc35'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:10.959759 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-eafafd5a-a5e4-4c9c-b086-ce489d504f88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4298d7435b4442fb88f1a823dc3fc35'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:10.967146 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-eafafd5a-a5e4-4c9c-b086-ce489d504f88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4298d7435b4442fb88f1a823dc3fc35'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:10.967755 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-eafafd5a-a5e4-4c9c-b086-ce489d504f88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4298d7435b4442fb88f1a823dc3fc35'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:10.968353 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-eafafd5a-a5e4-4c9c-b086-ce489d504f88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4298d7435b4442fb88f1a823dc3fc35'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:10.985595 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1466/2923] 10.222.0.39 () {58 vars in 1224 bytes} [Mon Aug 8 02:49:10 2022] DELETE /identity/v3/users/a4298d7435b4442fb88f1a823dc3fc35 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:49:11.157320 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33f8e0-1976-4426-b11a-d338b50d9a99 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:49:11.159133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8d33f8e0-1976-4426-b11a-d338b50d9a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:49:11.173035 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33f8e0-1976-4426-b11a-d338b50d9a99 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:49:11.229863 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ca922-3cbf-40e7-9b13-867f54889362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d716e3acbb54b2fa8e10b30ba8eb5fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '520cd26c01a04fe5b710bc3899ff38a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:49:11.230508 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ca922-3cbf-40e7-9b13-867f54889362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:49:11.230784 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ca922-3cbf-40e7-9b13-867f54889362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:49:11.231049 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ca922-3cbf-40e7-9b13-867f54889362 admin admin] PATH_INFO: `/v3/projects/1d68ad9b87794bb89b098a97c04ce073` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:49:11.232137 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ca922-3cbf-40e7-9b13-867f54889362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d68ad9b87794bb89b098a97c04ce073)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:49:11.234027 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ca922-3cbf-40e7-9b13-867f54889362 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:49:11.279133 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc5ca922-3cbf-40e7-9b13-867f54889362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d68ad9b87794bb89b098a97c04ce073', 'request_id': 'req-fc5ca922-3cbf-40e7-9b13-867f54889362'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:11.284785 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc5ca922-3cbf-40e7-9b13-867f54889362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d68ad9b87794bb89b098a97c04ce073', 'request_id': 'req-fc5ca922-3cbf-40e7-9b13-867f54889362'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:11.292882 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc5ca922-3cbf-40e7-9b13-867f54889362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d68ad9b87794bb89b098a97c04ce073', 'request_id': 'req-fc5ca922-3cbf-40e7-9b13-867f54889362'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:11.303836 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc5ca922-3cbf-40e7-9b13-867f54889362 admin admin] The token cache is being invalidate because project 1d68ad9b87794bb89b098a97c04ce073 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:49:11.304128 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc5ca922-3cbf-40e7-9b13-867f54889362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:11.304787 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc5ca922-3cbf-40e7-9b13-867f54889362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:49:11.306412 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1458/2924] 10.222.0.39 () {58 vars in 1233 bytes} [Mon Aug 8 02:49:11 2022] DELETE /identity/v3/projects/1d68ad9b87794bb89b098a97c04ce073 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:49:20.982182 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:49:21.300662 dv-cinder-733062-33-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=68,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:49:50.334216 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafafd5a-a5e4-4c9c-b086-ce489d504f88 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:49:50.335682 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-eafafd5a-a5e4-4c9c-b086-ce489d504f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:49:50.416606 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafafd5a-a5e4-4c9c-b086-ce489d504f88 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:49:50.477476 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8086584-e2e4-4d0f-82a3-c5e1c2b2c61f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f62745b92a864480a9438de581e1d05e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d474b61f1fe4f17aacb7a3665315c79', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:49:50.478081 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8086584-e2e4-4d0f-82a3-c5e1c2b2c61f service placement] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:49:50.478393 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8086584-e2e4-4d0f-82a3-c5e1c2b2c61f service placement] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:49:50.478666 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8086584-e2e4-4d0f-82a3-c5e1c2b2c61f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:49:50.480381 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e8086584-e2e4-4d0f-82a3-c5e1c2b2c61f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:49:50.491917 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8086584-e2e4-4d0f-82a3-c5e1c2b2c61f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:49:50.493485 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8086584-e2e4-4d0f-82a3-c5e1c2b2c61f service placement] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:49:50.504233 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1467/2925] 10.222.0.39 () {60 vars in 1365 bytes} [Mon Aug 8 02:49:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:50:00.499175 dv-cinder-733062-33-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=58 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}